Interface and mode selection for digital action execution

ABSTRACT

Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a U.S. National Stage under 35 U.S.C. § 371 ofInternational Patent Application No. PCT/US20/20341, filed Feb. 28, 2020and designating the United States, which is hereby incorporated byreference herein in its entirety.

BACKGROUND

A computing device can execute an application. The application canprovide a user interface that can receive input information, perform afunction, and output information.

SUMMARY

This technical solution is generally directed to selecting a type ofinterface and mode for executing digital actions. The technical solutioncan adjust the interface or mode based on a type of device andhistorical engagement level between a device and a digital assistant.For example, different types of interfaces or modes of execution canutilize different amounts of computation resources, energy or powerresources, network bandwidth resources, display resources, sensorresources, or time. Thus, it can be inefficient to select an interfacethat consumes excessive resources relative to a resource that utilizesfewer resources. However, certain resources may not be compatible withcertain types of devices, or may otherwise be inefficient to execute ina particular computing environment, thereby resulting in a potentiallymore efficient interface ultimately consuming excessive resources inorder to complete execution of a digital action. Therefore, systems andmethods of the present technical solution can select a type of interfaceor mode of execution of digital actions in order to reduce excessiveresource consumption, thereby improving the efficiency in which acomputing environment executes a digital action.

At least one aspect of the present technical solution is directed to asystem to adjust execution of digital actions. The system can include adata processing system including one or more processors. The dataprocessing system can load a script library embedded in an electronicresource. The data processing system can load the script library via aclient computing device. The client device can be one of multiple clientcomputing devices linked to an electronic account. The script librarycan include one or more call-to-actions for the electronic resource thatare configured for execution by one or more digital assistants providedby each of the client computing devices. The data processing system canquery a digital assistant component to determine a historic level ofengagement between the client computing devices and the one or moredigital assistants. The data processing system can select, based on afirst property of the client computing device and the historic level ofengagement, a type of digital interface in which to present acall-to-action of the plurality of call-to-actions. The data processingsystem can generate, based on the type of digital interface selectedbased on the first property of the client computing device and thehistoric level of engagement, a digital interface with thecall-to-action. The data processing system can detect, via the digitalinterface, an instruction to execute the call-to-action. The dataprocessing system can determine, responsive to the instruction toexecute the call-to-action, a mode of execution for the call-to-actionbased on a second property of the client computing device and thehistoric level of engagement. The data processing system can select,based on the mode of execution, a digital assistant from the one or moredigital assistants and a second client device of the plurality of clientcomputing devices to execute the call-to-action. The data processingsystem can transmit the call-to-action to the second client device tocause the second client device to invoke the digital assistant toexecute the call-to-action.

At least one aspect of the present technical solution is directed to amethod of adjusting execution of digital actions. The method can beperformed by a data processing system including one or more processors.The method can include the data processing system loading, via a clientcomputing device of a plurality of client computing devices linked to anelectronic account, a script library embedded in an electronic resource.The script library can include call-to-actions for the electronicresource configured for execution by one or more digital assistantsprovided by each of the client computing devices. The method can includethe data processing system querying a digital assistant component todetermine a historic level of engagement between the plurality of clientcomputing devices and the one or more digital assistants. The method caninclude the data processing system selecting, based on a first propertyof the client computing device and the historic level of engagement, atype of digital interface in which to present a call-to-action of theplurality of call-to-actions. The method can include the data processingsystem generating, based on the type of digital interface selected basedon the first property of the client computing device and the historiclevel of engagement, a digital interface with the call-to-action. Themethod can include the data processing system detecting, via the digitalinterface, an instruction to execute the call-to-action. The method caninclude the data processing system determining, responsive to theinstruction to execute the call-to-action, a mode of execution for thecall-to-action based on a second property of the client computing deviceand the historic level of engagement. The method can include the dataprocessing system selecting, based on the mode of execution, a digitalassistant from the one or more digital assistants and a second clientdevice of the plurality of client computing devices to execute thecall-to-action. The method can include the data processing systemtransmitting the call-to-action to the second client device to cause thesecond client device to invoke the digital assistant to execute thecall-to-action.

These and other aspects and implementations are discussed in detailbelow. The foregoing information and the following detailed descriptioninclude illustrative examples of various aspects and implementations,and provide an overview or framework for understanding the nature andcharacter of the claimed aspects and implementations. The drawingsprovide illustration and a further understanding of the various aspectsand implementations, and are incorporated in and constitute a part ofthis specification.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are not intended to be drawn to scale. Likereference numbers and designations in the various drawings indicate likeelements. For purposes of clarity, not every component may be labeled inevery drawing. In the drawings:

FIG. 1 is an illustration of a system to adjust execution of digitalactions, in accordance with an implementation.

FIG. 2 is an illustration of a process to adjust execution of digitalactions, in accordance with an implementation.

FIG. 3 is an illustration of a process to adjust execution of digitalactions, in accordance with an implementation.

FIG. 4 is an illustration of a graphical user interface for execution ofdigital actions, in accordance with an implementation.

FIG. 5 is an illustration of a graphical user interface for execution ofdigital actions, in accordance with an implementation.

FIG. 6 is an illustration of a method of adjusting execution of digitalactions, in accordance with an implementation.

FIG. 7 is a block diagram illustrating a general architecture for acomputer system that can be employed to implement elements of the systemdepicted in FIG. 1 , the processes and methods depicted in FIGS. 2, 3and 6 , and the graphical user interface elements depicted in FIGS. 4and 5 .

DETAILED DESCRIPTION

Following below are more detailed descriptions of various conceptsrelated to, and implementations of, methods, apparatus, and systems ofinterface and mode selection for digital action execution. For example,the methods, apparatus, and systems can adjust execution of digitalactions. The various concepts introduced above and discussed in greaterdetail below may be implemented in any of numerous ways.

This technical solution is generally directed to selecting an interfaceor mode of execution of digital actions. The systems and methods of thetechnical solution can adjust execution of digital actions by selectinga type of interface or mode of execution for the digital action.

Actions can be linked for execution by a voice-based digital assistant.An action (or function) can refer to or include any type of digitalaction that can be performed by one or more applications, computingdevices, servers, or electronic resources. For third-party (“3P”)applications or electronic resources that can interface with a digitalassistant, it can be challenging or not possible to efficiently providea notification of the functions the digital assistant can perform on theapplication. As the number of types of actions a digital assistant canperform are continually increasing, it can become increasinglychallenging and inefficient to discover such functions, let aloneproperly execute the function at an optimum time or manner in a givencontext.

For example, a notification can include displaying or playing audioindicating a type of function or action a digital assistant can perform.However, displaying or playing audio listing all the available digitalassistant functions with regard to a 3P application can be timeconsuming, inefficient, introduce delays, and provide a subpar orwasteful user interface as many of those functions may be irrelevant ina current computing environment or context. A system that lacksawareness of the relevant functions a digital assistant can perform on a3P application or electronic resource can result in the digitalassistant or application being underutilized or inefficiently utilizedby performing unnecessary or multiple actions that may otherwise beperformed by a more efficient digital assistant function. Thus, systemsand methods of the present technical solution can select an interface ormode of execution that can improve the efficiency in which an action canbe performed (e.g., by selecting a digital assistant function oraction).

The present technical solution can facilitate identifying and selectingavailable and relevant functions that a digital assistant with avoice-based interface can perform on a 3P application, and facilitateexecution of the relevant functions. For example, the system can captureavailable actions or functions a digital assistant can perform on aparticular application. The system can identify which actions arerelevant to a computing environment context. The system can provide anindication (e.g., a call-to-action) of those relevant actions via a userinterface. The system can immediately execute those relevant actionsfrom within the application, linking the call-to-action directly to therelevant action on the digital assistant (e.g., action-linking).

For example, a user can navigate to a website such as an electronicfinancial instrument trading website. The system can capture or detectthe contextual computing information (e.g., actions associated with thewebsite such as logging in, authenticating an account in order todetermine a current portfolio value, trade status, or execute a trade ortransfer of funds.) The system can identify available actions relevantto the current computing environment that a voice-based digitalassistant is configured to perform. The system can provide an indication(e.g., call-to-action) via a user interface directly within the websitethat provides a notification of the available digital assistant actionsrelevant to the current computing environment. The system can receive ordetect an instruction, command, or interaction with one of the digitalassistant actions presented via the notification. The system can providea digital assistant interface can facilitate execution or invocation ofthe selected action. Thus, this technical solution identifies andselects relevant actions on a third-party application or electronicresource that a digital assistant can perform more efficiently relativeto the actions being performed directly by the 3P application orelectronic resource without the digital assistant. The system candiscover that the digital assistant is configured to perform one or moreactions associated with the application or electronic resource (e.g.website) when the application is invoked, identify which actions arerelevant to the current computing environment, display call-to-actionsproviding notifications of the relevant actions, and then execute therelevant actions.

The system can provide dynamic interface adjustment for call-to-actiondiscovery. For 3P applications or electronic resources that providefunctions through a voice interface, it can be challenging ornot-possible to efficiently provide a notification of the functions thatthe application or resource provides in a manner that is non-disruptiveto the current non-voice computing interaction with the application orresource. The system can provide a platform-specific action linkingmechanism for a voice-based digital assistant. For 3P applications orelectronic resources that provide functions through a voice interface,it can be challenging or not possible to allow for efficient invocationof the 3P-provided voice interface functions from the non-voice contextin which the user is notified about the functions.

For example, upon discovering the available actions relevant to thecurrent computing environment context, the system can determine whattype of call-to-action (notification of available actions) to provide tothe user. The system can include a dynamic notifier component that candynamically determine (e.g., based on a parameter, characteristic orfeature associated with a current computing environment) a level ofengagement with the application or electronic resource, in order todetermine which call-to-actions to display, as well as the format ortype of the call-to-action. For example, if a client device is activelyinteracting with a website to perform certain digital actions, then thedynamic notifier of this technical solution can provide a pop-up windowwith the discovered call-to-action because the device (or user thereof)may be more likely to engage with a digital assistant to execute thecorresponding action. By providing a more conspicuous notification(e.g., relative to an embedded notification or banner notification orsmall icon located at a corner of the website), the dynamic notifier canincrease the likelihood of engaging with the digital assistant toexecute the action, thereby preventing the execution of functions via anon-voice-based interface directly with the application that may consumeadditional computing resources or waste time.

The system can determine that a use of the client device is passivelyviewing an application or electronic resource. Passively viewing anelectronic resource can refer to a non-interactive state or viewing anelectronic resource without interacting with the electronic resource.Not interacting with an electronic resource can refer to the clientdevice not receiving input directed to the electronic resource for aduration of time. Examples of applications or electronic resources thatcan be passively viewed can include music, videos, streaming music,streaming video, or other types of audio or video content that can beviewed or consumed without interaction for a duration. Upon detectingthat an electronic resource is being passively viewed, the dynamicnotifier can select a different format or type of notification that canbe less conspicuous as compared to the type of notification that can beselected for an application or electronic resource that is beingactively consumed. Examples of a less conspicuous notification caninclude an audio or voice overlay, or a small icon positioned in thecorner of a display screen or application graphical user interface.

When the system receives an interaction with a call-to-action on athird-party application or electronic resource, the system can performplatform-specific action linking for a voice-based digital assistant.Platform-specific action linking can refer to the system determining anappropriate platform-specific technology for directing or navigating theuser to the selected action within the digital assistant. The system caninclude a digital assistant selector component (e.g., a dynamic deviceselector) that can dynamically select a device to use to execute thedigital action. The system can identify multiple devices that are linkedto one another for the purposes of executing a digital action. Thedevice that displayed the notification and received the instruction toexecute the call-to-action (e.g., digital action) can be linked to anaccount that can be associated or linked with multiple devices. One ormore of the devices that are linked to the account can be configured toor capable of performing or executing the digital action. The deviceslinked to the account can be configured with digital assistants. One ormore of the devices can be available to execute the action responsive tothe instruction or during some other desired time interval. The systemcan use a selection policy to identify or select a device for executingthe digital action. The selection policy can select the device based ona current computing environment or context.

An example of platform-specific action linking can include using a deeplink to launch or trigger a native digital assistant to execute on theclient device to perform the action. In another example, the system cangenerate a universal application link or custom uniform resource locatorto launch a non-native digital assistant application on the clientdevice to perform the action. In yet another example, the system canselect a different device with a digital assistant to perform theaction. Platform-specific action linking can refer to or includeexecuting an action based on a type of operating system, type of device,characteristic of a device, configuration of a device, or capability ofa device, for example.

Accordingly, and in some cases, the system can dynamically determine amode of execution based on a configuration, type of device, applicationsinstalled on the device, network connections, or other characteristicsassociated with the computing environment. For example, if a digitalassistant application is installed, the system can navigate to thedigital assistant application and input a command to execute the digitalaction responsive to the call-to-action. If, on the other hand, if adigital assistant application is not installed on the device, the systemcan provide a notification requesting installation of the digitalassistant application prior to execution of the digital action. Thenotification can include a link, path, or pointer to a cloud-basedrepository of application executable files that can be downloaded andinstalled by the device.

The system can launch a digital assistant on a different device toexecute the digital action. For example, the electronic resource caninclude a webpage that is rendered in an application such as a webbrowser. The call-to-action can be presented via the web browser. Thecall-to-action can be overlaid on the electronic resource as a banner,pop-up, interstitial, or in some other manner. The system can detect aninteraction with the call-to-action or other indication to execute thedigital action corresponding to the call-to-action. The system candetermine current computing environment information, such as the type ofweb browser, device, network connectivity, account informationassociated with the device executing the application, or additionaldevices linked to the account. The system can, via a dynamic deviceselector component or functionality, display a web-page with a list ofdevices linked to a same account as the device that provided thecall-to-action. The system can pre-process or filter the list of thedevices such that the devices that are presented are only those devicesthat are compatible or capable of executing the digital actioncorresponding to the call-to-action. In some cases, the system can rankthis list of devices to weight devices based on relevancy, capability,compatibility, performance, or other factors. For example, the systemcan rank or filter the devices based on relevancy to the digital action(e.g., only show devices with screens for actions that may providegraphical output). The dynamic device selector of the present technicalsolution can rank the list of devices such that devices that are mostrelevant to a current computing context are ranked higher than devicesthat are less relevant to the current computing context (e.g., rankrecently used devices higher than devices that have not been usedrecently, rank frequently used devices higher than devices that are usedinfrequently).

FIG. 1 illustrates an example system 100 that selects interfaces andmodes for execution of a digital action. The system 100 can adjustexecution of digital actions. The system 100 can include contentselection infrastructure. The system 100 can include a data processingsystem 102. The data processing system 102 can include or execute on acomputing device 122. The data processing system 102 can communicatewith one or more of a 3P digital content provider device 134, remotedata processing system 136, or 3P electronic resource server 142 148 viaa network 105. The network 105 can include computer networks such as theInternet, local, wide, metro, or other area networks, intranets,satellite networks, and other communication networks such as voice ordata mobile telephone networks. The network 105 can be used to accessinformation resources such as web pages, web sites, domain names, oruniform resource locators that can be presented, output, rendered, ordisplayed on at least one computing device 122, such as a laptop,desktop, tablet, personal digital assistant, smart phone, portablecomputers, or speaker. For example, via the network 105 a user of thecomputing device 122 can access information or data provided by a 3Pdigital content provider device 134. The computing device 122 may or maynot include a display; for example, the computing device may includelimited types of user interfaces, such as a microphone and speaker. Insome cases, the primary user interface of the computing device 122 maybe a microphone and speaker. The computing device 122 can interface withor be included in a voice-based computing environment.

The network 105 can be used by the data processing system 102 to accessinformation resources such as applications, web pages, web sites, domainnames, or uniform resource locators that can be presented, output,rendered, or displayed by the client computing device 122. For example,via the network 105 a user of the client computing device 122 can accessinformation or data provided by the 3P digital content provider device134. The network 105 can include or constitute a sub-network ofinformation resources available on the internet that are associated witha content placement or search engine results system, or that areeligible to include third party digital components as part of a digitalcomponent placement campaign.

The network 105 may be any type or form of network and may include anyof the following: a point-to-point network, a broadcast network, a widearea network, a local area network, a telecommunications network, a datacommunication network, a computer network, an ATM (Asynchronous TransferMode) network, a SONET (Synchronous Optical Network) network, a SDH(Synchronous Digital Hierarchy) network, a wireless network and awireline network. The network 105 may include a wireless link, such asan infrared channel or satellite band. The topology of the network 105may include a bus, star, or ring network topology. The network mayinclude mobile telephone networks using any protocol or protocols usedto communicate among mobile devices, including advanced mobile phoneprotocol (“AMPS”), time division multiple access (“TDMA”), code-divisionmultiple access (“CDMA”), global system for mobile communication(“GSM”), general packet radio services (“GPRS”) or universal mobiletelecommunications system (“UMTS”). Different types of data may betransmitted via different protocols, or the same types of data may betransmitted via different protocols.

The system 100 can include, interface with, communicate with, orotherwise access a 3P electronic resource server 142. The dataprocessing system 102 can communicate with the 3P electronic resourceserver 142 via network 105. The 3P electronic resource server 142 can beremote from, and different from, the data processing system 102, remotedata processing system 136, 3P digital content provider device 134, andcomputing device 122. The 3P electronic resource server 142 can beassociated with a developer of the electronic resource 112. The 3Pelectronic resource server 142 can facilitate execution of theelectronic resource 112. For example, the 3P electronic resource server142 can perform backend processing for the electronic resource 112executed by application 110. The application 110 executing on thecomputing device 122 can execute a frontend component of the application110, and the 3P electronic resource server 142 can execute a backendcomponent of the application 110. The application 110 executing on thecomputing device 122 can transmit remote procedure calls or otherrequests or data to the 3P electronic resource server 142. The 3Pelectronic resource server 142 can transmit data, information, orrequests to the application 110 to perform a function or execute anaction. The 3P electronic resource server 142 can modify or change astate of the application 110 or electronic resource 112 executed orprovided via the application 110 executing on the computing device 122.

The system 100 can include at least one remote data processing system136. The remote data processing system 136 can include at least onelogic device such as a computing device having a processor tocommunicate via the network 105, for example with the data processingsystem 102, 3P digital content provider device 134 (e.g., contentprovider). The remote data processing system 136 can include at leastone computation resource, server, processor or memory. For example, theremote data processing system 136 can include a plurality of computationresources or servers located in at least one data center. The remotedata processing system 136 can include multiple, logically-groupedservers and facilitate distributed computing techniques. The logicalgroup of servers may be referred to as a data center, server farm or amachine farm. The servers can also be geographically dispersed. A datacenter or machine farm may be administered as a single entity, or themachine farm can include a plurality of machine farms. The serverswithin each machine farm can be heterogeneous - one or more of theservers or machines can operate according to one or more type ofoperating system platform.

Servers in the machine farm can be stored in high-density rack systems,along with associated storage systems, and located in an enterprise datacenter. For example, consolidating the servers in this way may improvesystem manageability, data security, the physical security of thesystem, and system performance by locating servers and high performancestorage systems on localized high performance networks. Centralizationof all or some of the remote data processing system 136 components,including servers and storage systems, and coupling them with advancedsystem management tools allows more efficient use of server resources,which saves power and processing requirements and reduces bandwidthusage.

The remote data processing system 136 can include a digital assistantserver 138 and a content selector component 140. The digital assistantserver 138 can be designed, constructed and operational to perform oneor more function in communication with the digital assistant component106 of the computing device 122. The content selector component 140 canbe designed, constructed and operational to select digital componentitems (e.g., content items) provided by the 3P digital content providerdevice 134. The content selector component 140 can select content itemsresponsive to a request for content from the computing device 122. Thecontent selector component 140 can transmit selected content items tothe computing device 122 for presentation (e.g., audio output, visualoutput, or audiovisual output).

The system 100 can include, access, or otherwise interact with at leastone 3P digital content provider device 134. The 3P digital contentprovider device 134 can include at least one logic device such as acomputing device having a processor to communicate via the network 105,for example with the computing device 122, the data processing system102, or the remote data processing system 136. The 3P digital contentprovider device 134 can include at least one computation resource,server, processor or memory. For example, the 3P digital contentprovider device 134 can include a plurality of computation resources orservers located in at least one data center. The 3P digital contentprovider device 134 can include or refer to a service provider device,or goods provider device.

The 3P digital content provider device 134 can provide audio baseddigital components for presentation by the computing device 122 as anaudio output digital component. The digital component can includeresponses to search queries or requests. The digital component caninclude information from databases, search engines, or networkedresources. For example, the digital component can include newsinformation, weather information, sports information, encyclopediaentries, dictionary entries, or information from digital textbooks. Thedigital component can include offers for a good or service, such as avoice based message that states: “Would you like me to order you ataxi?” The 3P digital content provider device 134 can include memory tostore a series of audio digital components that can be provided inresponse to a voice based query. The 3P digital content provider device134 can also provide audio based digital components (or other digitalcomponents) to the remote data processing system 136 where they can bestored for selection by the content selector component 140. The remotedata processing system 136 can select the audio digital components andprovide (or instruct the content provider computing device 140 toprovide) the audio digital components to the client computing device122. The audio based digital components can be exclusively audio or canbe combined with text, image, or video data. The digital components orcontent items can include images, text, video, multimedia, or othertypes of content in one or more formats.

The remote data processing system 136 can include a content placementsystem having at least one computation resource or server. The remotedata processing system 136 can include, interface, or otherwisecommunicate with at least one content selector component 140. The remotedata processing system 136 can include, interface, or otherwisecommunicate with at least one digital assistant server 138.

The content selector component 140 and digital assistant server 138 caneach include at least one processing unit or other logic device such asprogrammable logic array engine, or module configured to communicatewith one another or other resources or databases. The content selectorcomponent 140 and digital assistant server 138 can be separatecomponents, a single component, or part of the remote data processingsystem 136. The system 100 and its components, such as the remote dataprocessing system 136, can include hardware elements, such as one ormore processors, logic devices, or circuits.

The remote data processing system 136 can obtain anonymous computernetwork activity information associated with a plurality of computingdevices 122. A user of a computing device 122 can affirmativelyauthorize the remote data processing system 136 to obtain networkactivity information corresponding to the user’s computing device 122.For example, the remote data processing system 136 can prompt the userof the computing device 122 for consent to obtain one or more types ofnetwork activity information. The identity of the user of the computingdevice 122 can remain anonymous and the computing device 122 can beassociated with a unique identifier (e.g., a unique identifier for theuser or the computing device provided by the data processing system or auser of the computing device). The remote data processing system 136 canassociate each observation with a corresponding unique identifier.

A 3P digital content provider device 134 can establish an electroniccontent campaign. The electronic content campaign can be stored ascontent data in a data repository of the content selector component 140.An electronic content campaign can refer to one or more content groupsthat correspond to a common theme. A content campaign can include ahierarchical data structure that includes content groups, digitalcomponent data objects, and content selection criteria. To create acontent campaign, 3P digital content provider device 134 can specifyvalues for campaign level parameters of the content campaign. Thecampaign level parameters can include, for example, a campaign name, apreferred content network for placing digital component objects, a valueof resources to be used for the content campaign, start and end datesfor the content campaign, a duration for the content campaign, aschedule for digital component object placements, language, geographicallocations, type of computing devices on which to provide digitalcomponent objects. In some cases, an impression can refer to when adigital component object is fetched from its source (e.g., remote dataprocessing system 136 or 3P digital content provider device 134), and iscountable. In some cases, due to the possibility of click fraud, roboticactivity can be filtered and excluded, as an impression. Thus, in somecases, an impression can refer to a measurement of responses from a Webserver to a page request from a browser, which is filtered from roboticactivity and error codes, and is recorded at a point as close aspossible to opportunity to render the digital component object fordisplay on the computing device 122. In some cases, an impression canrefer to a viewable or audible impression; e.g., the digital componentobject is at least partially (e.g., 20%, 30%, 30%, 40%, 50%, 60%, 70%,or more) viewable on a display device of the client computing device122, or audible via a speaker of the computing device 122. A click orselection can refer to a user interaction with the digital componentobject, such as a voice response to an audible impression, amouse-click, touch interaction, gesture, shake, audio interaction, orkeyboard click. A conversion can refer to a user taking a desired actionwith respect to the digital component objection; e.g., purchasing aproduct or service, completing a survey, visiting a physical storecorresponding to the digital component, or completing an electronictransaction.

The 3P digital content provider device 134 can further establish one ormore content groups for a content campaign. A content group includes oneor more digital component objects and corresponding content selectioncriteria, such as keywords, words, terms, phrases, geographic locations,type of computing device, time of day, interest, topic, or vertical.Content groups under the same content campaign can share the samecampaign level parameters, but may have tailored specifications forparticular content group level parameters, such as keywords, negativekeywords (e.g., that block placement of the digital component in thepresence of the negative keyword on main content), bids for keywords, orparameters associated with the bid or content campaign.

To create a new content group, the 3P digital content provider device134 can provide values for the content group level parameters of thecontent group. The content group level parameters include, for example,a content group name or content group theme, and bids for differentcontent placement opportunities (e.g., automatic placement or managedplacement) or outcomes (e.g., clicks, impressions, or conversions). Acontent group name or content group theme can be one or more terms thatthe 3P digital content provider device 134 can use to capture a topic orsubject matter for which digital component objects of the content groupis to be selected for display. For example, a car dealership can createa different content group for each brand of vehicle it carries, and mayfurther create a different content group for each model of vehicle itcarries. Examples of the content group themes that the car dealershipcan use include, for example, “Make A sports car” “Make B sports car,”“Make C sedan,” “Make C truck,” “Make C hybrid,” or “Make D hybrid.” Anexample content campaign theme can be “hybrid” and include contentgroups for both “Make C hybrid” and “Make D hybrid”, for example.

The 3P digital content provider device 134 can provide one or morekeywords and digital component objects to each content group. Keywordscan include terms that are relevant to the product or services ofassociated with or identified by the digital component objects. Akeyword can include one or more terms or phrases. For example, the cardealership can include “sports car,” “V-6 engine,” “four-wheel drive,”“fuel efficiency,” as keywords for a content group or content campaign.In some cases, negative keywords can be specified by the contentprovider to avoid, prevent, block, or disable content placement oncertain terms or keywords. The content provider can specify a type ofmatching, such as exact match, phrase match, or broad match, used toselect digital component objects.

The 3P digital content provider device 134 can provide one or morekeywords to be used by the remote data processing system 136 to select adigital component object provided by the 3P digital content providerdevice 134. The 3P digital content provider device 134 can identify oneor more keywords to bid on, and further provide bid amounts for variouskeywords. The 3P digital content provider device 134 can provideadditional content selection criteria to be used by the remote dataprocessing system 136 to select digital component objects. Multiple 3Pdigital content provider devices 134 can bid on the same or differentkeywords, and the remote data processing system 136 can run a contentselection process or ad auction responsive to receiving an indication ofa keyword of an electronic message.

The 3P digital content provider device 134 can provide one or moredigital component objects for selection by the remote data processingsystem 136. The remote data processing system 136 (e.g., via contentselector component 140) can select the digital component objects when acontent placement opportunity becomes available that matches theresource allocation, content schedule, maximum bids, keywords, and otherselection criteria specified for the content group. Different types ofdigital component objects can be included in a content group, such as avoice digital component, audio digital component, a text digitalcomponent, an image digital component, video digital component,multimedia digital component, or digital component link. Upon selectinga digital component, the data processing system 102 can transmit thedigital component object for presentation via the computing device 122,rendering on a computing device 122 or display device of the computingdevice 122. Rendering can include displaying the digital component on adisplay device, or playing the digital component via a speaker of thecomputing device 122. The remote data processing system 136 can provideinstructions to a computing device 122 to render the digital componentobject. The remote data processing system 136 can instruct the digitalassistant component 106 of the computing device 122, or an audio driver132 of the computing device 122, to generate audio signals or acousticwaves. The remote data processing system 136 can instruct theapplication 110 executed by the computing device 122 to present theselected digital component object. For example, the application 110 caninclude a slot (e.g., a content slot) in which the digital componentobject can be presented (e.g., audio slot or visual slot).

The data processing system 102 can include, execute, or otherwisecommunicate with a content selector component 140 to receive thequeries, keywords or trigger keyword identified by the natural languageprocessor and select, based on the trigger keyword, a digital component.The content selector component 140 can select the digital component viaa real-time content selection process. The content selection process caninclude, for example, performing a search via a search engine, oraccessing a database stored on a remote server or device such as a 3Pdigital content provider device 134. The content selection process canrefer to, or include, selecting sponsored digital component objectsprovided by third party content providers 134. The real-time contentselection process can include a service in which digital componentsprovided by multiple content providers are parsed, processed, weighted,or matched in order to select one or more digital components to provideto the computing device 122. The content selector component 140 canperform the content selection process in real-time. Performing thecontent selection process in real-time can refer to performing thecontent selection process responsive to the request for content receivedvia the client computing device 122. The real-time content selectionprocess can be performed (e.g., initiated or completed) within a timeinterval of receiving the request (e.g., 1 second, 2 seconds, 5 seconds,10 seconds, 20 seconds, 30 seconds, 1 minute, 2 minutes, 3 minutes, 5minutes, 10 minutes, or 20 minutes). The real-time content selectionprocess can be performed during a communication session with the clientcomputing device 122, or within a time interval after the communicationsession is terminated. The data processing system 102 can select anaudio chime to provide subsequent to completion of the digital componentselection process or other digital task.

For example, the data processing system 102 can include a contentselector component 140 designed, constructed, configured or operationalto select digital component objects. To select digital components fordisplay in a voice-based environment, the data processing system 102(e.g., via an NLP component of the digital assistant component 106) canparse the input audio signal to identify queries, keywords (e.g., atrigger keyword), and use the keywords to select a matching digitalcomponent. The data processing system 102 can select the matchingdigital component based on a broad match, exact match, or phrase match.For example, the content selector component 140 can analyze, parse, orotherwise process subject matter of candidate digital components todetermine whether the subject matter of the candidate digital componentscorrespond to the subject matter of the keywords or phrases of the inputaudio signal detected by the microphone of the client computing device122. The content selector component 140 may identify, analyze, orrecognize voice, audio, terms, characters, text, symbols, or images ofthe candidate digital components using an image processing technique,character recognition technique, natural language processing technique,or database lookup. The candidate digital components may includemetadata indicative of the subject matter of the candidate digitalcomponents, in which case the content selector component 140 may processthe metadata to determine whether the subject matter of the candidatedigital component corresponds to the input audio signal.

3P digital content providers 134 may provide additional indicators whensetting up a content campaign that includes digital components. Thecontent provider may provide information at the content campaign orcontent group level that the content selector component 140 may identifyby performing a lookup using information about the candidate digitalcomponent. For example, the candidate digital component may include aunique identifier, which may map to a content group, content campaign,or content provider. The content selector component 140 may determine,based on information stored in content campaign data structure in datarepository 108, information about the 3P digital content provider device134.

The remote data processing system 136 can receive, via a computernetwork, a request for content for presentation on a computing device122. The data processing system 102 can identify the request byprocessing an input audio signal detected by a microphone of the clientcomputing device 122. The request can include selection criteria of therequest, such as the device type, location, and a keyword associatedwith the request.

Responsive to the request, the remote data processing system 136 canselect a digital component object from a data repository or a databasethat can include content provided by one or more 3P digital contentprovider device s134, and provide the digital component for presentationvia the computing device 122 via network 105. The computing device 122can interact with the digital component object. The computing device 122can receive an audio response to the digital component. The computingdevice 122 can receive an indication to select a hyperlink or otherbutton associated with the digital component object that causes orallows the computing device 122 to identify a service provider, requesta service from the service provider, instruct the service provider toperform a service, transmit information to the service provider, orotherwise query the service provider device.

The computing device 122 (or client computing device or client device)can include or execute a data processing system 102. The data processingsystem 102 can include, interface, or otherwise communicate with atleast one interface 104. The data processing system 102 can include,interface, or otherwise communicate with at least one local digitalassistant component 106 (or digital assistant component 106). The localdigital assistant component 106 can include natural language processingcapabilities or functionalities. The local digital assistant component106 can include a natural language processor (“NLP”) component. The dataprocessing system 102 can include, interface, or otherwise communicatewith at least one data repository 108. The data processing system 102can include or execute at least one application 110. The data processingsystem 102 can at least partially execute the at least one application110 (e.g., the application can include a client component and a servercomponent). The application 110 can execute or provide an electronicresource 112. The electronic resource 112 can include a script library114. The script library 114 can include, access, provide or otherwiseutilize an dynamic notifier component 116, or digital assistant selectorcomponent 120.

The data repository 108 can include one or more local or distributeddatabases, and can include a database management system. The datarepository 108 can include computer data storage or memory and can storeapplications 110, application data, profiles, indexes, deviceconfiguration information, account information, preference, dataassociated with the electronic resource, among other data. Applications110 can include an executable file, application package file,configuration file, or other data that facilitates execution of theapplication.

The interface 104, digital assistant component 106, application 110, orother component of the data processing system 102 can each include orutilize at least one processing unit or other logic device such asprogrammable logic array engine, or module configured to communicatewith one another or other resources or databases. The interface 104,digital assistant component 106, application 110 or other components ofthe data processing system 102 can be separate components, a singlecomponent, or part of the data processing system 102. The system 100 andits components, such as the data processing system 102, can includehardware elements, such as one or more processors, logic devices, orcircuits. Components, systems or modules of the data processing system102 can be executed at least partially by the remote data processingsystem 136.

The computing device 122 can include, interface, or otherwisecommunicate with at least one sensor 130, transducer 126, audio driver132, pre-processor 124, or display device 128. The sensor 130 caninclude, for example, an ambient light sensor, proximity sensor,temperature sensor, accelerometer, gyroscope, motion detector, GPSsensor, location sensor, microphone, or touch sensor. The transducer 126can include a speaker or a microphone. The audio driver 132 can providea software interface to the hardware transducer 126. The audio drivercan execute the audio file or other instructions provided by the dataprocessing system 102 to control the transducer 126 to generate acorresponding acoustic wave or sound wave. The display device 128 caninclude one or more component or functionality of display 735 depictedin FIG. 7 . The pre-processor 124 can be configured to detect a triggerkeyword, predetermined hot word, initiation keyword, or activationkeyword. In some cases, the trigger keyword can include a request toperform an action. In some cases, the trigger keyword can includepredetermined action keyword to enable or activate the computing device122, and the request keywords can follow the trigger keyword or hotword. The pre-processor 124 can be configured to detect a keyword andperform an action based on the keyword. The pre-processor 124 can detecta wake-up word or other keyword or hotword, and responsive to thedetection, invoke the digital assistant component 106 of the dataprocessing system 102 executed by computing device 122. In some cases,the pre-processor 124 can filter out one or more terms or modify theterms prior to transmitting the terms to the remote data processingsystem 136 for further processing. The pre-processor 124 can convert theanalog audio signals detected by the microphone into a digital audiosignal, and transmit or provide one or more data packets carrying thedigital audio signal to the data processing system 102 or the remotedata processing system 136 via the network 105. In some cases, thepre-processor 124 can provide, to the digital assistant component 106 orthe remote data processing system 136, data packets carrying some or allof the input audio signal, responsive to detecting an instruction toperform such transmission. The instruction can include, for example, atrigger keyword or other keyword or approval to transmit data packetscomprising the input audio signal to the data processing system 102 orthe remote data processing system 136.

The client computing device 122 can be associated with an end user thatenters voice queries as audio input into the client computing device 122(via the sensor 130) and receives audio output in the form of a computergenerated voice that can be provided from the data processing system 102(or the 3P digital content provider device 134) to the client computingdevice 122, output from the transducer 126 (e.g., a speaker). Thecomputer generated voice can include recordings from a real person orcomputer generated language.

The data processing system 102 can include an interface 104 designed,configured, constructed, or operational to receive and transmitinformation using, for example, data packets. The interface 104 canreceive and transmit information using one or more protocols, such as anetwork protocol. The interface 104 can include a hardware interface,software interface, wired interface, or wireless interface. Theinterface 104 can facilitate translating or formatting data from oneformat to another format. For example, the interface 104 can include anapplication programming interface that includes definitions forcommunicating between various components, such as software components.The interface 104 can facilitate communications between one or morecomponent of the system 100, such as between the digital assistantcomponent 106, the application 110, and the data repository 108.

The data processing system 102 can include a local digital assistantcomponent 106. The local digital assistant component 106 can include anapplication, script or program installed at the client computing device122. The local digital assistant component 106 can receive inputsignals, data packets or other information. The local digital assistantcomponent 106 can receive data packets or other input via interface 104.The local digital assistant 106 can include an application to receiveinput audio signals from the interface 104 of the data processing system102, and to drive components of the client computing device to renderoutput audio signals. The data processing system 102 can receive datapackets or other signals that include or identify an audio input signal.For example, the local digital assistant component 106 can be configuredwith NLP techniques, functionality or components that can receive orobtain the audio signal and parse the audio signal. The digitalassistant component 106 can provide for interactions between a human anda computer. The digital assistant component 106 can be configured withtechniques for understanding natural language and allowing the dataprocessing system 102 to derive meaning from human or natural languageinput. The digital assistant component 106 can include or be configuredwith technique based on machine learning, such as statistical machinelearning. The digital assistant component 106 can utilize decisiontrees, statistical models, or probabilistic models to parse the inputaudio signal. The digital assistant component 106 can perform, forexample, functions such as named entity recognition (e.g., given astream of text, determine which items in the text map to proper names,such as people or places, and what the type of each such name is, suchas person, location, or organization), natural language generation(e.g., convert information from computer databases or semantic intentsinto understandable human language), natural language understanding(e.g., convert text into more formal representations such as first-orderlogic structures that a computer module can manipulate), machinetranslation (e.g., automatically translate text from one human languageto another), morphological segmentation (e.g., separating words intoindividual morphemes and identify the class of the morphemes, which canbe challenging based on the complexity of the morphology or structure ofthe words of the language being considered), question answering (e.g.,determining an answer to a human-language question, which can bespecific or open-ended), semantic processing (e.g., processing that canoccur after identifying a word and encoding its meaning in order torelate the identified word to other words with similar meanings).

The digital assistant component 106 can (e.g., utilizing an NLPtechnique, functionality or component) convert the audio input signalinto recognized text by comparing the input signal against a stored,representative set of audio waveforms and choosing the closest matches.The set of audio waveforms can be stored in data repository 108 or otherdatabase accessible to the data processing system 102. Therepresentative waveforms can be generated across a large set of users,and then may be augmented with speech samples from the user. After theaudio signal is converted into recognized text, the digital assistantcomponent 106 can match the text to words that are associated, forexample via using a model stored in data repository 108 that has beentrained across users or through manual specification, with actions thatthe data processing system 102 can serve.

The audio input signal can be detected by the sensor 130 or transducer126 (e.g., a microphone) of the client computing device 122. Via thetransducer 126, the audio driver 132, or other components the clientcomputing device 122 can provide the audio input signal to the dataprocessing system 102, where it can be received (e.g., by the interface104) and provided to the local digital assistant 106 or stored in thedata repository 108.

The digital assistant component 106 can obtain the input audio signal.From the input audio signal, the digital assistant component 106 canidentify at least one request or at least one trigger keywordcorresponding to the request. The request can indicate intent or subjectmatter of the input audio signal. The trigger keyword can indicate atype of action likely to be taken. For example, the digital assistantcomponent 106 can parse the input audio signal to identify at least onerequest to invoke an application. The digital assistant component 106can parse the input audio signal to identify at least one request, suchas a request to leave home for the evening to attend dinner and a movie.The keyword can include at least one word, phrase, root or partial word,or derivative indicating an action to be taken. For example, the keyword“go” or “to go to” from the input audio signal can indicate a need fortransport. In this example, the input audio signal (or the identifiedrequest) does not directly express an intent for transport, however thekeyword indicates that transport is an ancillary action to at least oneother action that is indicated by the request.

The digital assistant component 106 can parse the input audio signal toidentify, determine, retrieve, or otherwise obtain the request and thekeyword. For instance, the digital assistant component 106 can apply asemantic processing technique to the input audio signal to identify thekeyword or the request. The digital assistant component 106 can applythe semantic processing technique to the input audio signal to identifyone or more keywords. A keyword can include one or more terms or aphrase. The digital assistant component 106 can apply the semanticprocessing technique to identify an intent to perform a digital action.

For example, the computing device 122 can receive an input audio signaldetected by a sensor 130 (e.g., microphone) of the client computingdevice 122. The input audio signal can be “Digital assistant, I needsomeone to do my laundry and my dry cleaning.” A pre-processor 124 ofthe client computing device 122 can detect a wakeup word, hot word, ortrigger keyword in the input audio signal, such as “Digital assistant”.The pre-processor 124 can detect the wakeup word, hot word, or triggerkeyword by comparing an audio signature or waveform in the input audiosignal with a model audio signature or waveform that corresponds to thetrigger keyword. The pre-processor 124 can determine that the inputaudio signal includes a wakeup word, hot word, or trigger keyword thatindicates that the input audio signal is to be processed by the digitalassistant component 106. Responsive to detecting hot word, wakeup word,or trigger keyword, the pre-processor 124 can determine, authorize,route, forward, or other provide the detected input audio signal to thedata processing system 102 for processing by the digital assistantcomponent 106.

The digital assistant component 106 can receive the input audio signaland apply a semantic processing technique, or other natural languageprocessing technique, to the input audio signal comprising the sentenceto identify trigger phrases “do my laundry” and “do my dry cleaning”. Insome cases, the digital assistant component 106 can provide data packetscorresponding to the input audio signal to the remote data processingsystem 136 to cause the digital assistant server 138 to process theinput audio signal. The digital assistant component 106 can process theinput audio signal in conjunction with or via the digital assistantserver 138. The digital assistant component 106 can further identifymultiple keywords, such as laundry, and dry cleaning.

The digital assistant component 106 can identify search queries,keywords, intents or phrases corresponding to performing a search orother request for information. The digital assistant component 106 candetermine that the input audio signal corresponds to a request forinformation about a topic, event, current event, news event, dictionarydefinition, historical event, person, place or thing. For example, thedigital assistant component 106 can determine that the input audiosignal corresponds to a query, request, intent, or action to make travelarrangements, book a ride, obtain information, perform a web search,check stock prices, launch an application, check the news, order food,or shop for other products, goods or services.

The digital assistant component 106 can use one or more techniques toparse or process the input audio signal. Techniques can includerule-based techniques or statistical techniques. Techniques can utilizemachine learning or deep learning. Example techniques can include namedentity recognition, sentiment analysis, text summarization, aspectmining, or topic mining. Techniques can include or be based on textembeddings (e.g., real values vector representations of strings),machine translation (e.g., language analysis and language generation),or dialogue and conversations (e.g., models used by artificialintelligence). Techniques can include determine or leverage syntaxtechniques (e.g., arrangement of words in a sentence based on grammar)such as lemmatization, morphological segmentation, word segmentation,part-of-speech tagging, parsing, sentence breaking, or stemming.Techniques can include determine or leverage semantics techniques suchas named entity recognition (e.g., determining the parts of the textthat can be identified and categorized into present groups such as namesof application 110, people or places), word sense disambiguation, ornatural language generation.

In some cases, the digital assistant component 106 can launch theapplication 110. In some cases the application 110 may already belaunched prior to the digital assistant component 106 receiving theinput audio signal. For example, based on processing or parsing theinput audio signal, the digital assistant component 106 can identify anapplication 110 to invoke, launch, open, or otherwise activate. Thedigital assistant component 106 can identify the application 110 basedon parsing the input audio signal to identify the terms, keywords,trigger keywords or phrases. The digital assistant component 106 canperform a lookup in the data repository 108 using the identified terms,keywords, trigger keywords or phrases to identify the application 110.In some cases, the keyword can include an identifier of the application110, such as “Application_Name_A” or “Application_Name_B”. In somecases, the keyword can indicate a type or category of application 110,such as ride sharing application, restaurant booking application, movieticket application, news application, weather application, navigationapplication, streaming music application, streaming video application,restaurant review application, or other type or category of application110. For cases in which the application 110 may already be launched andexecuting prior to receipt of the input audio signal, the digitalassistant component 106 can process the input audio signal to determinean action to perform in the application 110 or responsive to acall-to-action presented via the electronic resource 112 rendered by theapplication 110.

The client computing device 122 can execute an application 110. The dataprocessing system 102 can execute an application 110. The dataprocessing system 102 can include or execute an operating system viawhich the client computing device 122 can execute the application 110.The application 110 can include any type of application the clientcomputing device 122 is configured to execute, run, launch, or otherwiseprovide. The application 110 can include a multimedia application, musicplayer, video player, web browser, word processor, mobile application,desktop application, tablet application, electronic game, an electroniccommerce application, or other type of application 110. The application110 can execute, render, load, parse, process, present or otherwiseoutput data corresponding to an electronic resource 112. An electronicresource 112 can include, for example, a web site, web page, multimediaweb content, video content, audio content, travel content, entertainmentcontent, content related to shopping for goods or services, or othercontent.

The application 110 can receive data associated with the electronicresource 112 from a third-party (“3P”) electronic resource server 142.The 3P electronic resource server 142 can provide the electronicresource 112 for execution by the application 110. The 3P electronicresource server 142 can include a file server, web server, gamingserver, multimedia server, cloud computing environment, or other backendcomputing system configured to provide data to cause the application 110to present or provide the electronic resource 112 via the computingdevice 122. The computing device 122 can access the 3P electronicresource server 142 via network 105.

An administrator of the 3P electronic resource server 142 can develop,establish, maintain or provide the electronic resource 112. The 3Pelectronic resource server 142 can transmit the electronic resource 112to the computing device 122 responsive to a request for the electronicresource 112. The electronic resource 112 can be associated with anidentifier, such as a uniform resource locator (“URL”), uniform resourceidentifier, web address, or filename, or file path. The 3P electronicresource server 142 can receive the request for the electronic resource112 from the application 110. The electronic resource 112 can include anelectronic document, webpage, multimedia content, streaming content,audio, video, text, images, video game, or other digital or electroniccontent

The administrator of the 3P electronic resource server 142 can provide ascript library 114. The 3P electronic resource server 142 can embed thescript library 114 in the electronic resource 112. The script library114 can include a JavaScript library. The script library 114 can beconfigured to communicate with the remote data processing system 136 toprovide digital assistant-based functionality for the electronicresource 112. The script library 114 can provide user interface elements(e.g., buttons, forms, autocomplete suggestion) on the electronicresource 112.

The data processing system 102 can load a graphical user interfaceelement corresponding to the script library 114 at a location on theelectronic resource 112 established by a provider of the electronicresource. The script library 114 can be launched in an inline frame(“iframe”) on the electronic resource 112. By launching or executing inan iframe on the electronic resource 112, the data processing system 102can host the script library 114, or corresponding user interfaceelements, in a secure computing environment. The computing environmentcan be secure with reference to the electronic resource or the 3Pelectronic resource server 142. For example, the iframe in which thescript library 114 executes can control or prevent access to thecontents or data of the iframe. The iframe can prevent the electronicresource 112 or the 3P electronic resource server 142 from accessingcontents or data associated with the script library 114. The scriptlibrary 114 executing in the iframe can communicate data with thedigital assistant server 138 without the electronic resource 112 (or 3Pelectronic resource server 142) obtaining access to the data. The scriptlibrary 114 can obtain data or information associated with the dataprocessing system 102, an account logged into the data processing system102, the computing device 122 or other information.

The administrator of the electronic resource 112 can place hypertextmarkup language (“HTML”) markup on the electronic resource 112. Theelectronic resource 112 can include HTML markup. The HTML markup on theelectronic resource 112 can include description or configurationsettings for the manner in which the script library is to display userinterface elements on the electronic resource 112. The followingillustrates an example of HTML markup of an electronic resource 112:

   <digital-assistant-action-link-group>          <digital-assistant-action-link                 title = “Market updates”                  href = “https://assistant.digital.com/services/invoke/uid/000000d139bbc4d4” >             </digital-assistant-action-link>             <digital-assistant-action-link             title = “What’s my balance?”             href = “https://assistant.digital.com/services/invoke/uid/000000d139bbc4d4” >              </digital-assistant-action-link>   </digital-assistant-action-link-group>

The customized web element <digital-assistant-action-link-group> can actas a container, holding call-to-actions that the 3P electronic resourceserver 142 provides for display on the electronic resource 112. Each<digital-assistant-action-link> within the container can represent asingle call-to-action to be displayed on the electronic resource 112.The 3P can specific additional configuration for the call-to-actions onthe page, such as icons, a project identifier, or attributes. Forexample, an attribute can be a hypertext reference (“href”), and have atype “string”. The href attribute can include an action link uniformresource locator “URL” to link into a native digital assistantexperience. The URL can be validated before rendering the selectedaction user interface. The data processing system 102 can block or notrender invalid action links. To validate the action link, the dataprocessing system 102 or remote data processing system 136 can: 1)determine whether the loaded script library 114 suffix matches the URLsuffix of the action link; 2) determine that the loaded script library114 has an action for the intent of the action link; and 3) determinewhether the loaded script library 114 has links enabled for the intentof the action link.

Another example attribute can include “title” and have a type “string”.The title attribute can display or provide a title. The title can be thesame as the intent_title of the action link, or a different title set byan administrator of the electronic resource 112.

The application 110, such as a web browser, can load the electronicresource 112. Loading the electronic resource 112 can includedownloading the data or contents of the electronic resource 112 from the3P electronic resource server 142 or other file server or web server orhost server. Loading the electronic resource 112 can include theapplication 110 rendering or executing the electronic resource 112.Loading the electronic resource 112 can include the application 110loading the script library 114 embedded with the electronic resource112. The script library can include multiple call-to-actions for theelectronic resource 112 configured for execution by one or more digitalassistants. The call-to-actions can be configured for execution by alocal digital assistant component 106 or a digital assistant server 138.The call-to-actions can be executed by both the local digital assistantcomponent 106 and the digital assistant server 138. For example, thelocal digital assistant component 106 and the digital assistant server138 can communicate with one another to execute the call-to-actions, orexecute different portions of the call-to-action.

The computing device 122, or data processing system 102, can beassociated or linked with multiple digital assistants that are executedby multiple computing devices 122. The digital assistants can beassociated or linked with the same account identifier. For example, auser can log into the data processing system 102 with a username andpassword, or other credentials. The data processing system 102 canauthenticate and authorize the user to utilize a digital assistantcomponent. The account can be associated with multiple computing devices122, such as a smartphone, laptop computer, tablet computer, smartwatch,and desktop computer. Each of the computing devices 122 can include alocal digital assistant component 106 that can interface with a digitalassistant server 138. Thus, the script library can be configured withcall-to-actions that can be executed by one or more digital assistantscorresponding to the one or more local digital assistant components 106executed by one or more computing devices 122 that are linked orassociated with the same account identifier of a user of the computingdevice 122.

Upon loading the electronic resource 112, the application 110 can causethe script library 114 to load in a secure iframe of the electronicresource 112. The script library 114 can launch, invoke, initiate,execute, call or otherwise provide one or more components, such as andynamic notifier component 116, and a digital assistant selectorcomponent 120. The components of the script library 114 can communicatewith one or more component or element of the data processing system 102or remote data processing system 136 in a secure manner, such as withoutthe 3P electronic resource server 142 obtaining the communicated data.The script library can be established by a third-party service provider(e.g., a 3P electronic resource server 142) with call-to-actions thatinclude a predetermined set of call-to-actions selected by thethird-party service provider. An administrator of 3P electronic resourceserver 142 can establish call-to-actions for the electronic resource112, and embed the call-to-actions in the script library 114. Eachcall-to-action can be associated with attributes or metadata thatindicate what task the call-to-action or when, or under what conditions,to invoke or present the call-to-action. The script library 114 can bedesigned, configured and operational to perform one or more functions,processes or methods depicted in FIG. 2 and FIG. 3 , such as processes200 and 300.

The dynamic notifier component 116 can be designed, constructed andoperational to query the digital assistant component 106 to determine anhistoric level of engagement between computing device 122 and one ormore digital assistants. The dynamic notifier component 116 candetermine the historic level of engagement between digital assistantsand an account associated with the computing device 122. The account canbe associated with multiple computing devices. In some cases, thehistorical level of engagement can be based on engagement with a digitalassistant on all the computing devices 122 associated with the sameaccount. In some cases, the historic level of engagement can be based onengagement with the digital assistant associated with the computingdevice 122 that loaded the script library 114.

The dynamic notifier component 116 can determine what call-to-action topresent, and then select a type of digital interface with which topresent the call-to-action. To select the call-to-action, the dynamicnotifier component 116 can determine a current context or state of theelectronic resource 112. For example, the state or context can refer toor include a search state, purchase state, checkout flow, or informationgathering flow. The context can indicate that a user is searching forflights, or was unable to identify flights with a desired price, time,or duration, for example. The electronic resource 112 can present asurvey or prompt to obtain context information. The dynamic notifiercomponent 116 can identify a call-to-action associated with a currentcontext or state of the electronic resource 112.

In order to select a type of interface to generate, the dynamic notifiercomponent 116 can determine the historic level of engagement. Historicallevel engagement can refer to a previous level of engagement. Historiclevel of engagement can refer to the level of engagement prior to thecurrent time. Historical level of engagement can refer to the level ofengagement in the last 24 hours, 48 hours, 72 hours, 1 week, 2 weeks, 3weeks, 4 weeks, 30 days, 60 days, 90 days, 6 months, or more.

The data processing system 102 (e.g., via dynamic notifier component116) can receive, via a network 105 from a digital assistant server 138remote from the data processing system 102, information about thehistoric level of engagement. The dynamic notifier component 116 candetermine the level of engagement based on an amount of use of a digitalassistant. Level of engagement can be based on a number of uses of adigital assistant, number of interactions with a digital assistant, ornumber of times a digital assistant has been launched or invoked. Levelof engagement can be based on the types of interactions with a digitalassistant. Types of interactions can include, for example, categories ortypes of tasks performed by the digital assistant, types of queries,types of requests, or types of call-to-actions. Example types of tasksor call-to-actions can include launching an application, performing anonline search, making a purchase, ordering a service, a ride sharerequest, checking the weather or other type of task. Categories of tasksor call-to-actions can include, for example, requests for information,entertainment tasks, purchases, commerce, rideshare requests, ormultimedia requests. The dynamic notifier component 116 can determinethe historic level of engagement based on a number of previousinteractions between the plurality of client computing devices during atime interval (e.g., last 24 hours, 48 hours, 7 days, or 30 days).

The dynamic notifier component 116 can determine the level of engagementfor a particular category or task. Level of engagement can be determinedfor a particular electronic resource 112 or application 110. Level ofengagement can be determined for a particular time of day, day of theweek, or location of the computing device 122. Level of engagement canbe determined for a particular account logged into the computing device122. Level of engagement can be determined for a particular user of thecomputing device 122.

The dynamic notifier component 116 can determine the level of engagementas a score, value, percentage, or other label. For example, the dynamicnotifier component 116 can determine the level of engagement as one oflow, medium, or high. The dynamic notifier component 116 can determine anumeric value or score to represent the level of score. For example, thedynamic notifier component 116 can use a scale from 1 to 10 to representthe level of engagement, with 1 being the lowest level of engagement,and 10 being the highest level of engagement. In another example, thedynamic notifier component 116 can represent the level of engagement asa grade, such as A, B, C, D or F, with A being the highest level ofengagement, and F being the lowest level of engagement. The dynamicnotifier component 116 can use a binary value to represent the level ofengagement, with 1 indicating that there has been historic engagementbetween the account and a digital assistant during a previous timeinterval (e.g., last 1 week), and 0 indicating that there has been noengagement between the account and digital assistant or that the accountwas recently created.

The level of engagement can be based on multiple factors. The multiplefactors can be combined to determine an overall level of engagement.Factors can include, for example, types of engagement, categories ofengagement, or other aspects of historic engagement. For example, thedynamic notifier component 116 can assign a value for multipleengagement factors, and then combine the values (e.g., sum, product,average, or weighted average) to determine an overall score. Forexample: number of engagements in past week + number of engagements withsame electronic resource 112 + number of engagements with same categoryof electronic resource 112 + number of engagements on same computingdevice 122. The dynamic notifier component 116 can determine an overallscore based on these factors. The dynamic notifier component 116 cancompare the score with a threshold to determine whether the level ofengagement is low or high; or map the score to a more granular levelssuch as low, medium-low, medium, medium-high, or high level ofengagement. The thresholds can be predetermined or configured by anadministrator of the remote data processing system 136. The thresholdcan be dynamically determined based on an aggregated level of engagementof other accounts.

For example, the electronic resource 112 can be a website directed topurchasing flight tickets. The dynamic notifier component 116 candetermine that a digital assistant was used to facilitate purchasingflight tickets on this electronic resource 112 in the past 30 days.Accordingly, the dynamic notifier component 116 can determine that thelevel of engagement for this account and this electronic resource 112 ishigh.

To determine the historic level of engagement, the dynamic notifiercomponent 116 can communicate with the digital assistant server 138 vianetwork 105. The dynamic notifier component 116 can transmit a requestor query to the digital assistant server 138 to obtain information fromthe digital assistant server 138 regarding a historic level ofengagement between the account logged into the computing device 122 anddigital assistant. In some cases, the dynamic notifier component 116 canobtain the information from the local digital assistant component 106.For example, the dynamic notifier component 116 can query or otherwisecommunicate with the local digital assistant component 106 to determinethe level of engagement. The local digital assistant component 106 canstore information relating to historic or previous engagements,interactions, uses, invocations, or use of the digital assistantcomponent 106 or a digital assistant 106 on other computing devicesassociated with the same account.

The dynamic notifier component 116 can access the information relatingto the historic level of engagement in a secure manner. For example, thescript library 114 can be launched in an iframe that can be a securecomputing environment or sandboxed computing environment that preventsthe electronic resource 112 from accessing data obtained or used by thedynamic notifier component 116. Thus, the dynamic notifier component 116can determine the historic level of engagement for the account withoutthe electronic resource 112 obtaining the information.

The dynamic notifier component 116 can determine a property of thecomputing device 122 (e.g., client computing device). Property of thecomputing device 122 can refer to or include a type of computing device,location of the computing device, configuration of the computing device,application installed on the computing device, operating system of thecomputing device, battery power remaining, performance capabilities, oravailable interfaces. For example, a property relating to type of devicecan include mobile device or desktop device, smartphone, tablet,desktop, smartwatch, wearable device, smart speaker, smart television,or an appliance. A property of the computing device 122 can indicatewhether or not the computing device 122 has a native digital assistant,a microphone, or speaker. A native digital assistant can refer to alocal digital assistant component 106 being integrated or part of theoperating system installed or executing on the computing device 122. Acomputing device 122 without a native digital assistant can have adigital assistant component installed as a third party application onthe computing device 122. In some cases, the computing device 122 maynot have any digital assistant installed, in which case the propertyvalue can indicate that no digital assistants are installed oravailable. A property can include whether an account associated with theelectronic resource has been logged into.

The dynamic notifier component 116 can determine device properties. Thedynamic notifier component 116 can obtain the device properties from theremote data processing system 136. For example, the remote dataprocessing system 136 can transmit device properties to the dataprocessing system 102. In some cases, the data processing system 102 canobtain device properties directly from the computing device 122.Properties can include, for example: account identifier; deviceoperating system type; device operating system version; or devicelocation.

The dynamic notifier component 116 can select a type of digitalinterface in which to present a call-to-action of the call-to-actionsincluded in the script library 114. The dynamic notifier component 116can select the type of digital interface based on a property of thecomputing device 122 and the historic level of engagement. Acall-to-action can correspond to a task associated with the electronicresource 112 that a digital assistant can perform. Example tasks caninclude ordering a ride from a ridesharing application, booking flightson a travel electronic resource, purchasing goods or services from acommerce electronic resource, follow-on or recurring tasks on theelectronic resource, or conducting a financial transaction, for example.

Types of digital interfaces in which to present a call-to-action caninclude an icon located in a corner of the electronic resource ordisplay device of the computing device 122, banner, pop-up window,overlay, animated icon, scrolling text, flashing icon, inline text orimage, or audio output. Types of digital interfaces can be conspicuousdigital interfaces (e.g., an overlay over the entire electronic resource112 or a majority of an area of the electronic resource 112), or asubtle or inconspicuous digital interface (e.g., an icon in a top rightcorner of the electronic resource 112 that does not obfuscate theremainder of the electronic resource).

The dynamic notifier component 116 can select the type of digitalinterface based on a property of the client computing device 122 and thehistoric level of engagement. The dynamic notifier component 116 can usevarious techniques to select the digital interface based on one or moreproperties and the historic level of engagement. For example, theselection techniques used by the dynamic notifier component 116 can useor include one or more of account preferences, predeterminedconfiguration, mapping, historical performance of digital interfaces, ormachine learning.

The dynamic notifier component 116 can use account preferences to selectthe digital interface. The account preferences can indicate a preferencefor a digital interface depending on the type of computing device. Forexample, a user of the computing device 122 can establish accountpreferences. The account preferences can include a ranking ofpreferences of types of digital interfaces for types of computing device122. For example, the ranking of digital interfaces for a smartphone canbe: 1) icon in corner of screen; 2) banner; and 3) overlay overelectronic resource, where 1 represent the highest rank (or mostpreferred type of digital interface) and 3 represent the lowest rank (orleast preferred type of digital interface). The dynamic notifiercomponent 116 can augment this ranking with the historic level ofengagement associated with the smartphone to select a type of digitalinterface. For example, if the historic level of engagement has beenhigh, then the dynamic notifier component 116 can select the highestranked type of digital interface. If the historic level of engagementhas been low, then the dynamic notifier component 116 can determine toselect a different type of digital interface in order to increase thelikelihood of engagement with the digital assistant. In another example,if the historic level of engagement with digital assistant is high, thenthe dynamic notifier component 116 can override the ranking to select atype of digital interface that is more conspicuous so as to increase thelikelihood of engagement with the digital assistant. Increasing thelikelihood of engagement with a digital assistant can reduce computingresource utilization because a digital assistant can increase efficiencyin performance of a computing task.

The dynamic notifier component 116 can utilize a predetermined mappingto select a type of digital interface. An example mapping is illustratedin Table 1.

TABLE 1 Illustrative Mapping of Property and Historic Level ofEngagement to Type of Digital Interface Property Historic Level ofEngagement Type of Digital Interface Smartphone with native digitalassistant Low Icon in corner of electronic resource Smartphone withnative digital assistant medium Banner in bottom of electronic resourcesmartphone with native digital assistant high Conspicuously overlayinterface over electronic resource Smartphone with no digital assistantinstalled low Icon in corner of electronic resource Smartphone with nodigital assistant installed medium Banner in bottom of electronicresource Smartphone with no digital assistant installed high Banner inbottom of electronic resource

The dynamic notifier component 116 can utilize a mapping provided by anadministrator of the digital assistant server 138 or 3P electronicresource server 142. The mapping can be referred to as a selectiontechnique, policy, or logic. The dynamic notifier component 116 candetermine the property and level of historic engagement to identify atype of digital interface to select. The dynamic notifier component 116can perform a lookup in the map using the property and level to identifythe corresponding type of digital interface.

The dynamic notifier component 116 can use historical performance ofdigital interfaces to determine a digital interface to select. Thedynamic notifier component 116 can obtain properties and levels ofhistoric engagement associated with the performance of digitalinterfaces. Performance of the digital interface can refer to whetherthe interface resulted in an interaction or execution of thecall-to-action. Positive performance can refer to executing thecall-to-action to cause engagement with a digital assistant, whereasnegative performance can refer to closing, terminating or otherwise notexecuting the call-to-action or not invoking or using a digitalassistant to perform a task. The dynamic notifier component 116 canutilize a model trained by a machine learning technique in which thefeatures of the model can include one or more properties, historic levelof engagement, type of digital interface, and performance of the digitalinterface (e.g., a binary value that indicates executing thecall-to-action or absence of execution of the call-to-action).

The dynamic notifier component 116 can use the machine learningtechnique to select the type of digital interface. One or morecomponents or systems can generate the model using the machine learningtechnique. For example, the remote data processing system 136 (e.g., viadigital assistant server 138) can generate the model, the local digitalassistant component 106 can generate the model, or the dynamic notifiercomponent 116 can generate the model. The data used to train a modelused by the machine learning using data for a particular account. Thedynamic notifier component 116 can generate or use a model for aparticular account using data associated with the account. In somecases, the dynamic notifier component 116 can aggregate de-identified oranonymized data associated with multiple accounts to train a model.

The dynamic notifier component 116 can apply weights to scores or valuesassociated with properties or historic level of engagement to determinea score. The dynamic notifier component 116 can use the score to selecta type of digital interface. For example, the existence of lack of aproperty can be represented as a numeric value. The historic level ofengagement can be represented as a numeric value. The dynamic notifiercomponent 116 can weight the scores (e.g., apply a multiplier toscores). The dynamic notifier component 116 can combine the scores (orweighted scores) to determine an overall score used to select a digitalinterface. The dynamic notifier component 116 can map the score to atype of digital interface. An example score based mapping is illustratedTable 2.

Score Threshold Type of Digital Interface Less than 3 Icon in corner ofelectronic resource 3 to 5 Banner in bottom of electronic resourceGreater than 5 and less than 7 Conspicuously overlay interface overelectronic resource Greater than 7 Voice-based interface

Thus, the dynamic notifier component 116 can make a customized, dynamicselection of a type of digital interface based on real-time informationassociated with a property of a computing device 122 and a historiclevel of engagement with digital assistant in order to improve thecomputational efficiency with which the computing device 122, 3Pelectronic resource server 142, or remote data processing system 136 canexecute tasks or functions associated with the electronic resource 112.For example, the data processing system 102 (e.g., via dynamic notifiercomponent 116) can select the type of digital interface as a voice-basedinterface based on property of the client computing device 122indicating the client computing device 122 includes a microphone and aspeaker, and the historic level of engagement indicating a number ofinteractions between client computing devices 122 associated with thesame account and the one or more digital assistants greater than athreshold (e.g., 7).

The dynamic notifier component 116 can generate a digital interface withthe call-to-action. The dynamic notifier component 116 can generate thedigital interface based on the type of digital interface selected basedon the first property of the client computing device 122 and thehistoric level of engagement. The dynamic notifier component 116 cangenerate the digital interface responsive to the selection. The dynamicnotifier component 116 can use, call, populate, or otherwise utilize atemplate, images, text, scripts, functions, user interface elements,audio, multimedia or other components to generate the digital interface.For example, the dynamic notifier component 116 can access a templatefor a digital interface, and populate fields in the template based onthe call-to-action established by the 3P electronic resource server 142for the electronic resource 112. In another example, the script library114 can include pre-built or configured digital interfaces for each typeof digital interface. The dynamic notifier component 116 can render thepre-built or pre-configured digital interface that corresponds to theselected type of digital interface for the call-to-action. In anotherexample, the dynamic notifier component 116 can select the type ofdigital interface as a graphical user interface based on the firstproperty of the client computing device 122 indicating the clientcomputing device 122 lacks a microphone (or access to the microphone ora speaker of the client computing device 122 by the one or more digitalassistants has been disabled). If the access to the microphone isdisabled, then the dynamic notifier component 116 can select a type ofdigital interface that can receive non-voice-based input, such as touchinput, or keyboard or mouse input.

The data processing system 102 can detect an instruction to execute thecall-to-action. The instruction can be detected via an input device ofthe client computing device 122. The computing device 122 can receivethe input via a touch interface, mouse input, keyboard input, voiceinput, gesture based input, or other type of input. The input can be aninteraction with the digital interface, such selection of a button, linkor other interactive widget presented via the interface. Thus, the dataprocessing system 102 can detect, via the digital interface, aninstruction to execute the call-to-action.

The data processing system 102 or script library 114 can include orexecute a digital assistant selector component 120 designed, constructedand operational to determine a mode of execution for the call-to-action,select a digital assistant to execute the call-to-action, and transmitthe call-to-action. The digital assistant selector component 120 candetermine, responsive to the instruction to execute the call-to-action,a mode of execution for the call-to-action based on one or moreproperties of the client computing device 122 and the historic level ofengagement. The mode of execution can refer to or include executing anative digital assistant on the computing device 122, or executing anon-native digital assistant on the computing device 122. A non-nativedigital assistant can refer to or include a digital assistant that isnot pre-installed with an operating system. A non-native digitalassistant can refer to a digital assistant that is installed as athird-party application on the computing device 122.

Modes of execution can include, for example: navigating to a non-nativedigital assistant application installed on the computing device 122;redirecting to the electronic resource with a prompt or indication todownload and install the digital assistant; navigating to a nativedigital assistant installed on the computing device 122; navigating tothe native digital assistant and immediately executing thecall-to-action; navigating to a digital assistant electronic resourcewith a prompt indicating how to execute the call-to-action; providing aprompt with a list of other computing devices 122 having digitalassistants that are linked to the same account as the computing device122 executing the data processing system 102 and providing a prompt toselect one of the other computing devices 122 to launch a digitalassistant to execute the call-to-action.

The data processing system 102 can select the mode of execution for thecall-to-action based on one or more properties of the client computingdevice indicating a type of operating system of the client computingdevice. The property used by the data processing system 102 to selectthe mode of execution can be referred to as a second property or asecond one or more properties, while a property used to select thedigital interface can be referred to as a first property or a first oneor more properties. The first property can be the same as the secondproperties. A first one or more properties can overlap with a second oneor more properties. The first property can be different from the secondproperties. A first one or more properties can be mutually exclusivefrom the second one or more properties.

To select the mode of execution, the digital assistant selectorcomponent 120 can use one or more function or technique similar to thedynamic notifier component 116. For example, the digital assistantselector component 120 can utilize a mapping technique, index, machinelearning technique, logic-based technique, rules or policy.

The digital assistant selector component 120 can select a mode ofexecution based on the second property and historic level of engagement.The digital assistant selector component 120 can perform a lookup in amap or index with the second property and historic level of engagementto identify corresponding of mode of execution. The digital assistantselector component 120 can determine the mode of execution using amachine learning model or model trained with a machine learningtechnique. The model can include features such as properties, historiclevel of engagement, different modes of execution, and historicperformance of a digital assistant or execution of an action with thecorresponding mode of execution.

The digital assistant selector component 120 can select the mode ofexecution based on preferences associated with an account, orconfiguration provided by the 3P electronic resource server 142, ordigital assistant server 138.

The digital assistant selector component 120 can select, based on themode of execution, a digital assistant from the one or more digitalassistants and a second client device of the plurality of clientcomputing devices to execute the call-to-action. For example, the modeof execution can include launching, invoking, executing, or otherwiseutilizing a digital assistant on a second computing device 122 (e.g., acomputing device 122 that is different than the computing device 122that initially loaded the electronic resource 112 and script library114). The digital assistant selector component 120 can provide a list ofavailable computing devices 122, and receive a selection of a computingdevice. The digital assistant selector component 120 can rank a list ofcomputing devices 122 that are linked to the same account and configuredor operational to launch a digital assistant to execute thecall-to-action. The digital assistant selector component 120 canautomatically select the highest ranking computing device 122, orreceive an indication of a selection of a computing device.

For example, the list of computing devices 122 with digital assistantsconfigured to execute the call-to-action can be ranked based on one ormore of: how frequently the computing device 122 has been used to launcha digital assistant to execute a call-to-action; how recently thecomputing device 122 was last used; how close (location/proximity) thecomputing device 122 that initially loaded the script library 114 is tothe other computing device 122; or how relevant the computing device 122is to the call-to-action. For example, a digital assistant-enabled smartspeaker computing device 122 is highly relevant to a call-to-action forplaying music, whereas a smartwatch computing device 122 is lessrelevant to such a call-to-action. In another example, a digitalassistant-enabled smart television is highly relevant to acall-to-action for playing a video, whereas a smartwatch computingdevice 122 is less relevant to such a call-to-action.

The digital assistant selector component 120 can select a secondcomputing device 122 different from a first computing device (e.g., thecomputing device that initially loaded the script library 114) toexecute the call-to-action. The digital assistant selector component 120can transmit the call-to-action to the second client computing device122 to cause the second client computing device 122 to invoke thedigital assistant to execute the call-to-action. The digital assistantselector component 120 can transmit information that facilitateexecuting the call-to-action, such as information associated with theelectronic resource 112, or an account associated with the electronicresource 112. The information can include a reference, pointer,metadata, location information, or other data that facilitates executingthe call-to-action by the digital assistant of the second computingdevice 122.

In some cases, the system 100 can select digital components forprovision. The system can select content items or digital componentsfrom 3P digital content provider devices 134. The system 100 can providethe selected digital component for presentation via the computing device122, on the electronic resource 112, with the digital interface, or onthe second computing device 122 that executes the call-to-action. Thecontent item or digital component can be selected based on theelectronic resource 112, properties, historic level of engagement,call-to-action, or other information.

For example, the remote data processing system 136 can include a contentselector component 140. The content selector component 140 of the remotedata processing system 136 can select content items (e.g., digitalcomponent objects). The content selector component 140 can selectcontent items based on or responsive to one or more of a: request forcontent, based on the dynamic notifier component 116 selecting a digitalinterface, based on the digital assistant selector component 120selecting a mode of execution or computing device 122, based on thecall-to-action, or based on transmission of a call-to-action based onthe request or responsive to the request. The content selector component140 can select a content item that is relevant or otherwise matchesinformation associated with the call-to-action. For example, if theapplication is an e-commerce application that sells shoes, then thecontent selector component 140 can select a content item for a type ofsneaker. In another example, if the call-to-action relates to booking aflight, then the remote data processing system 136 can select a contentitem related to travel (e.g., a coupon for a flight).

In some cases, the content selector component 140 of the remote dataprocessing system 136 can determine to select a content item (e.g., adigital component object) provided by a 3P digital content providerdevice 134 subsequent to the dynamic notifier component 116 of the dataprocessing system 102 selecting a type of digital interface. Subsequentto the dynamic notifier component 116 selecting a type of digitalinterface, the dynamic notifier component 116 can provide an indicationof the selected type of digital interface to the remote data processingsystem 136. The dynamic notifier component 116 can provide theindication to the digital assistant server 138. The digital assistantserver 138 can generate a request for a content item and forward therequest to the content selector component 140. The digital assistantserver 138 can generate the request for content responsive to receivingthe indication of the selection of the type of digital interface. Insome cases, the digital assistant server 138 can determine to generatethe request for content based on the type of digital interface thedynamic notifier component 116 generated. For example, if the dynamicnotifier component 116 selects a prominent or conspicuous interface thatoccupies more screen real-estate, then the digital assistant server 138can determine that there may be sufficient room to include a contentitem. Therefore, the digital assistant server 138, responsive to thedetermination there is sufficient screen space for a supplementalcontent item, can generate the request for content. However, if theselected type of digital interface is more inconspicuous, such as asmall icon in the corner of the screen, then the digital assistantserver 138 can determine to not generate a request for content, therebyreducing wasted computing resource utilization.

Upon generating the request for content, the digital assistant server138 can forward the request for content to the content selectorcomponent 140. The content selector component 140 can perform areal-time content selection process to select a content item. Thecontent selector component 140 can use one or more parameters orselection criteria to select the content item. For example, the requestreceived by the content selector component 140 can include informationabout the electronic resource 112 (e.g., keywords, topics, concepts, orother contextual information), the call-to-action to be provided in adigital interface, the selected type of digital interface, orinformation associated with an electronic account. The content selectorcomponent 140 can use the information to select a relevant content item.

The remote data processing system 136 can provide the selected contentitem to the data processing system 102 for presentation. The remote dataprocessing system 136 can provide the content item to the script library114 to cause the script library 114 to display or present the contentitem via the digital interface selected by the dynamic notifiercomponent 116. The dynamic notifier component 116 can display thecontent item along with the call-to-action presented via the digitalinterface. Thus, the real-time content selection process can occursubsequent to selecting the type of digital interface and prior topresenting the digital interface for display.

In some cases, the data processing system 102 can present the contentitem via the digital assistant component 106. The remote data processingsystem 136 can provide the selected content item to the application 110for presentation. The remote data processing system 136 can provide theselected content item to the digital assistant component 106 to provideto the application 110 for presentation. The content item can bepresented via the user interface of the application 110, such as in avisual content slot or audio output. The dynamic notifier component 116can provide or present the content item along with the selected digitalinterface. The digital assistant selector component 120 can provide thecontent item to the second computing device 122 selected for executingthe call-to-action to cause the second computing device 122 to presentthe content item.

The digital assistant component 106 can present the content itemseparately or independently from the application 110. For example, thedigital assistant component 106 can invoke a separate user interface,such as a pop-up window or banner content for display via the displaydevice 128 of the computing device 122. In some cases, the digitalassistant component 106 can provide the content item as audio output.The audio output can be presented prior to, during, or subsequent toaudio output of the application 110. In the event the application 110 isnot providing audio output, the digital assistant component 106 canprovide the content item via audio output independent of the interfaceused by the application. Thus, the data processing system 102 canpresent the content item via a user interface of the digital assistantcomponent 106 (e.g., an audio interface) or the application 110 (e.g., agraphical user interface).

In some cases, the remote data processing system 136 can transmit theselected content item to a second computing device 122 different fromthe computing device 122 that initially loaded the electronic resource112. For example, the remote data processing system 136 can receive anindication from the digital assistant selector component 120 thatidentifies the second computing device 122. The digital assistantselector component 120 can select the second computing device 122 toexecute the call-to-action provided via the digital interface. Thus,rather than present the selected content item via the computing device122, the system 100 can present the content item via the secondcomputing device 122 that executes the call-to-action.

To do so, and in some cases, the remote data processing system 136 cantransmit the content item to the data processing system 102, and thedata processing system 102 can determine to delay presentation of thecontent item until the digital assistant selector component 120 selectsa digital assistant of a second computing device 122 to execute thecall-to-action. Upon selecting a digital assistant of the secondcomputing device 122, the data processing system 102 can forward theselected content item to the corresponding second computing device 122along with the call-to-action for execution.

In some cases, the data processing system 102 can present the contentitem along with the digital interface, and then transmit the contentitem to the selected digital assistant of the second computing device122 to cause the second computing device 122 to redisplay the selectedcontent item or a related content item.

FIG. 2 is an illustration of a process to adjust execution of digitalactions, in accordance with an implementation. The process 200 can beperformed by one or more system or component depicted in FIG. 1 or FIG.7 , including, for example, the data processing system, remote dataprocessing system, local digital assistant, script library, dynamicnotifier component, or digital selector component. The method 200 caninitiate by inspecting whether or an account has been activated for thesession. If an account has not been activated, then the data processingsystem can determine it may not be possible to dynamically select adigital interface because it may not be possible to determine thehistoric level of engagement. Accordingly, the data processing systemcan select a default digital interface if an account is not active orlogged in for the session. The default digital interface, when selected,can generate a prompt to request account login credentials.

If an account is logged-in, the script library can inspect whether ornot the account has engaged with a digital assistant by (1)communicating with a digital assistant server to confirm whether or notthere are any assistant-enabled devices associated with the account, or(2) whether the account has completed a digital assistant onboardingexperience in the past. If the account is not an Assistant user, theythe script library can select the same default, non-customized digitalinterface button as if the account was not logged in because there is nosignal with regard to historical level of engagement that can be used todynamically select a digital interface that is more prominent. If,however, the account is logged-in and the account has historicallyengaged with a digital assistant, then the library can inspectwhether/not there exist any past interactions between the user and thethird-party electronic resource provider via a digital assistant thatassociated with the account. The script library can determine thehistoric level of engagement with a digital assistant to perform anaction on the electronic resource by querying a digital assistantserver.

If the account has not interacted with the electronic resource using thedigital assistant in the past (e.g., if logged interactions between theelectronic resource and a digital assistant associated with the accountdoes not exist in the past), then the script library can select the samenon-personalized default digital interface button, as there is no signal(either positive or negative) as to whether they may benefit from more amore prominent call-to-action for digital assistant voice functions.

If the account is signed in, the account has engaged with a digitalassistant, and the account has engaged with the digital assistant tointeract with or perform an action on the electronic resource, thescript library can determine to show a more prominent call-to-action fordigital assistant voice functions on the electronic resource.

For example, at ACT 202, the data processing system executing the scriptlibrary can determine whether there is an account with an assistantprovider available. The account is not available (e.g., not signed in),then the script library can proceed to ACT 218 to display a defaultdigital interface that is not customized for the inline button entrypoint, and then proceed to ACT 220 to provide a web explore based actionlink such as a link with information about digital assistants, as wellas ACT 210 to provide a proactive bottom sheet entry point. If, however,at ACT 202 the script library determines that an account associated witha digital assistant is logged in for the session (e.g., via an operatingsystem of a computing device, application, or electronic resource), thescript library (or system executing the script library) can proceed toACT 204 to determine whether the account has engaged with a digitalassistant. At ACT 204, the script library can determine a historic levelof engagement with the account. The script library can determine whetherthe account has used digital assistants in the past.

If, at ACT 204, the script library determines that the account has notused digital assistants in the past, then the script library can proceedto ACT 218 to select the default digital interface for the inline buttonentry point. If the script library determines at ACT 204 that theaccount has engaged with digital assistants, the script library canproceed to ACT 206 to determine whether the account (or computing device122 on which the account is logged in) has used a digital assistant forthe electronic resource to execute the call-to-action. If, at ACT 206,the script library determines that the user has previously used thedigital assistant to execute an action for the electronic resource, thescript library can proceed to ACT 208 to select a digital interface thatcorresponds to this level of engagement and based on a property of thedevice, and then proceed to ACT 210. If, at ACT 206, the script librarydetermines that the account has not used a digital assistant to executean action for the electronic resource, the script library can proceed toACT 210 to provide a proactive bottom sheet entry point.

At ACT 210, the script library can provide a proactive bottom sheetentry point, which can include determining a mode of execution for thedigital assistant. The script library can determine to provide a moreprominent digital interface. The script library can proceed to ACT 212to determine whether an account link to the electronic resource isneeded. The account link can link the digital assistant to theelectronic resource to facilitate the digital assistant executing anaction associated with the electronic resource. The script library candetermine whether an account link is needed based on the type of actionto be executed (e.g., a purchase). If, at ACT 212, the script librarydetermines that an account link is needed, the script library canproceed to ACT 214 to execute an account link flow, which can include aprompt for account credentials or a request to link an account of theelectronic resource with the digital assistant, and then proceed to ACT216. If the script library determines at ACT 212 that an account link isnot needed, the script library can proceed to ACT 216 to provide anaction link user interface with a device selector. The script librarycan rank available devices that are configured with a digital assistantto execute the action, and receive a selection of the device to use toexecute the action. In some cases, the script library can automaticallyselect a highest ranking device to execute the action.

FIG. 3 is an illustration of a process to adjust execution of digitalactions, in accordance with an implementation. The process 300 can beperformed by one or more system or component depicted in FIG. 1 or FIG.7 , including, for example, the data processing system, remote dataprocessing system, local digital assistant, script library, dynamicnotifier component, or digital selector component. The process 300 canbe used for devices that already have linked accounts. For example, atACT 302, the script library can determine whether an account with adigital assistant provider is available. The script library candetermine whether a digital assistant account is activated or logged infor the session, such as in an application, operating system, or device.If, at ACT 302, the script library determines that an account with adigital assistant is not logged in or available, the script library canproceed to ACT 306 to select and provide an inline button entry point.The script library can provide a default digital interface for thecall-to-action that is not customized. The script library can proceed toACT 308 to provide a web explore based action link, which can refer toor include a link to an electronic resource.

If, however, at ACT 302 the script library determines a digitalassistant account is logged in, the script library can proceed to ACT304 to determine a historic level of engagement with digital assistants.If the account has not historically engaged with digital assistants, orhas a low level of engagement with the digital assistants that is belowa threshold (e.g., it has been more than 3 months since the accountengaged with a digital assistant), then the script library can proceedto ACT 306 to provide a default, non-customized digital interface.

If, at ACT 304, the script library determines the account has engagedwith digital assistants, or has engaged beyond a threshold, then thescript library can proceed to ACT 310 to determine whether an account ofthe electronic resource is already linked with the account of thedigital assistant. The script library can determine whether a digitalassistant account is linked to an account of a 3P electronic resource or3P electronic resource server. If a digital assistant account is notlinked to an account of the electronic resource, then the script librarycan proceed to ACT 318 to execute an account linking flow.

If, however, at ACT 310 the script library determines that the digitalassistant account is linked to an electronic resource account (e.g.,account of a travel website, music service, or online marketplace), thenthe script library can proceed to ACT 312 to determine whether thedevice or account has already used the digital assistant to execute thecall-to-action for the electronic resource. If the script librarydetermines at 312 that the device has not used digital assistant toexecute the action for the electronic resource, then the script librarycan proceed to ACT 316 to return an action link user interface (“UI”)with a device selector. If, however, the script library determines thatdigital assistants have been used to execute actions for the electronicresource, then the script library can proceed to ACT 314 to select aninline button entry point, which can be a customized or personalizeddigital interface that is prominently displayed with a call-to-actionfor a digital assistant. The script library can select a more prominentdigital interface based the account being linked and previously used toexecute an action for the electronic resource. The script library canthen proceed to ACT 318 to provide an action link UI with a deviceselector. The script library can rank the available devices forselection, or automatically select the highest ranking device.

FIG. 4 is an illustration of a graphical user interface for execution ofdigital actions, in accordance with an implementation. The graphicaluser interface 400 can be provided by one or more system or componentdepicted in FIG. 1 or FIG. 7 , including, for example, the dataprocessing system, remote data processing system, local digitalassistant, script library, dynamic notifier component, digital selectorcomponent, or display device. The graphical user interface 400 caninclude an electronic resource 112, main content 402 and an icon orinline entry point button 404. The main content 402 can include or referto text, images, multimedia content or other content that is provided bythe provider of the electronic resource 112. The icon 404 can be anexample of a digital interface type, and can include an inline buttonentry point with a prompt, such as “try it with digital assistant.” Ascript library can select the button 404 based on detecting that adigital assistant account is not logged in (e.g., via processes 200 or300). The script library can display the icon 404 below the main content402 so as not to obfuscate or otherwise block the main content 402.

FIG. 5 is an illustration of a graphical user interface for execution ofdigital actions, in accordance with an implementation. The graphicaluser interface 500 can be provided by one or more system or componentdepicted in FIG. 1 or FIG. 7 , including, for example, the dataprocessing system, remote data processing system, local digitalassistant, script library, dynamic notifier component, digital selectorcomponent, or display device. The graphical user interface 500 caninclude a display of the electronic resource 112, main content 402, apop-up window 502, with buttons to link an account 504 or not link anaccount 506. The graphical user interface 500 can be provided viaprocesses 300 or 400.

The pop-up window can be a digital interface and correspond to a type ofdigital interface selected by the script library based on properties anda historic level of engagement. The script library can select, generateand provide the pop-up window 502 responsive to determining that adigital assistant account is logged in, that the account has previouslyengaged with the digital assistant, but the account of the electronicresource provider is not linked with the digital assistant provider. Forexample, the script library can determine based on the historic level ofengagement and properties of the device to select a digital interfacetype with a call-to-action that is more prominent. For example if thefirst property indicates the client computing device is a smartphoneenabled with the digital assistant, and the historic level of engagementindicates that computing devices interacted with the one or more digitalassistants during a time interval prior to the script library loaded bythe data processing system for the electronic resource, then then thescript library can select the digital interface having a pop-up icon 502overlaid on the electronic resource 112. The pop-up window 502 is moreprominent that the icon 404 depicted in FIG. 4 because the pop-up window502 blocks a portion of the main content 402 of the electronic resource112 (e.g., half of the main content 402). The script library candetermine to provide a more prominent digital interface type. The linkaccount button 504 can execute an account linking flow, as depicted inprocesses 200 and 300.

FIG. 6 is an illustration of a method of adjusting execution of digitalactions, in accordance with an implementation. The method 600 can beperformed by one or more system or component depicted in FIG. 1 or FIG.7 , including, for example, the data processing system, remote dataprocessing system, local digital assistant, script library, dynamicnotifier component, or digital selector component.

At 602, the method 600 can include a computing device or data processingsystem loading a script library of or embedded in an electronicresource. The script library can be embedded in a webpage provided by aweb server. The data processing system or device can execute the scriptlibrary in an inline frame of the electronic resource. The scriptlibrary can include one or more call-to-actions, components, or modules.The data processing system can load, via a client computing devicelinked to an electronic account of a digital assistant, the scriptlibrary embedded in an electronic resource. The script library caninclude multiple call-to-actions for the electronic resource that areconfigured for execution by one or more digital assistants providedclient computing devices.

At 604, the script library can determine whether a digital assistantaccount is logged in for the session (e.g., currently logged in on thedevice). The script library can query an application or operating systemof the device to determine whether the digital assistant account isactive, activated or otherwise logged in. If the script librarydetermines the digital assistant account is not active or logged in, thescript library can proceed to ACT 606 to provide a default digitalinterface for the call-to-action that is not customized or configuredfor a digital assistant to execute.

If, however, at ACT 604 the script library determines that the digitalassistant account is active, then the script library can proceed to ACT608. At 608, the script library can determine a historic level ofengagement. The script library can query a digital assistant server todetermine a historic level of engagement between the account and digitalassistants. The script library can query the digital assistant server todetermine an historic level of engagement between the client computingdevices linked to the electronic account and the one or more digitalassistants.

At 610, the script library can select and generate a digital interfacebased on the historic level of engagement between the account anddigital assistant. The script library can select the digital interfacebased on a first property of the client computing device and thehistoric level of engagement. The script library can provide thecall-to-action via the digital interface.

At 612, the script library can detect an instruction to execute thecall-to-action, and determine a mode of execution for thecall-to-action. The script library can determine, responsive to theinstruction to execute the call-to-action, a mode of execution for thecall-to-action based on a second property of the client computing deviceand the historic level of engagement. The mode of execution can be, forexample, to launch a native digital assistant or a 3P provided digitalassistant.

At ACT 614, the script library can determine whether there are anycompatible devices available. The script library can identify availabledigital assistant-enabled devices. If the script library is unable toidentify any digital assistant-enabled devices that are currentlyavailable, then the script library can proceed to ACT 606 to provide adefault digital interface. For example, the mode of execution can be touse a native digital assistant, but the client device may not have anative digital assistant installed. In another example, the mode ofexecution can be to use a third party provided digital assistant, butthe current client device may not have the third party digital assistantinstalled, and a second linked device may be offline or not locatedproximate to the first computing device.

If, however, the script library identifies available digital assistantenabled devices that can execute the call-to-action via the selectedmode of execution, then the script library can proceed to ACT 616. AtACT 616, the script library can select a digital assistant to executethe call-to-action. The script library can select a second computingdevice based on one or more properties and the historic level ofengagement. For example, the script library can select a digitalassistant configured on a second computing device that was recently usedto execute a similar call-to-action and located proximate (e.g., on thesame WIFI network) as the first computing device. At ACT 618, the scriptlibrary can transmit the call-to-action to the selected second computingdevice to cause the digital assistant of the second computing device toexecute the call-to-action.

FIG. 7 is a block diagram of an example computer system 700. Thecomputer system or computing device 700 can include or be used toimplement the system 100, or its components such as the data processingsystem 102. The computing system 700 includes a bus 705 or othercommunication component for communicating information and a processor710 or processing circuit coupled to the bus 705 for processinginformation. The computing system 700 can also include one or moreprocessors 710 or processing circuits coupled to the bus for processinginformation. The computing system 700 also includes main memory 715,such as a random access memory (RAM) or other dynamic storage device,coupled to the bus 705 for storing information, and instructions to beexecuted by the processor 710. The main memory 715 can be or include adata repository. The main memory 715 can also be used for storingposition information, temporary variables, or other intermediateinformation during execution of instructions by the processor 710. Thecomputing system 700 may further include a read only memory (ROM) 720 orother static storage device coupled to the bus 705 for storing staticinformation and instructions for the processor 710. A storage device725, such as a solid state device, magnetic disk or optical disk, can becoupled to the bus 705 to persistently store information andinstructions. The storage device 725 can include or be part of the datarepository.

The computing system 700 may be coupled via the bus 705 to a display735, such as a liquid crystal display, or active matrix display, fordisplaying information to a user. An input device 730, such as akeyboard including alphanumeric and other keys, may be coupled to thebus 705 for communicating information and command selections to theprocessor 710. The input device 730 can include a touch screen display735. The input device 730 can also include a cursor control, such as amouse, a trackball, or cursor direction keys, for communicatingdirection information and command selections to the processor 710 andfor controlling cursor movement on the display 735. The display 735 canbe part of the data processing system 102, the client computing device122 or other component of FIG. 1 , for example.

The processes, systems and methods described herein can be implementedby the computing system 700 in response to the processor 710 executingan arrangement of instructions contained in main memory 715. Suchinstructions can be read into main memory 715 from anothercomputer-readable medium, such as the storage device 725. Execution ofthe arrangement of instructions contained in main memory 715 causes thecomputing system 700 to perform the illustrative processes describedherein. One or more processors in a multi-processing arrangement mayalso be employed to execute the instructions contained in main memory715. Hard-wired circuitry can be used in place of or in combination withsoftware instructions together with the systems and methods describedherein. Systems and methods described herein are not limited to anyspecific combination of hardware circuitry and software.

Although an example computing system has been described in FIG. 7 , thesubject matter including the operations described in this specificationcan be implemented in other types of digital electronic circuitry, or incomputer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them.

In situations in which the systems described herein collect personalinformation about users or applications installed on a user device, ormake use of personal information, the users are provided with anopportunity to control whether programs or features collect userinformation (e.g., information about a user’s social network, socialactions, or activities, profession, a user’s preferences, or a user’scurrent location). In addition or in the alternative, certain data canbe treated in one or more ways before it is stored or used, so thatpersonal information is removed.

The subject matter and the operations described in this specificationcan be implemented in digital electronic circuitry, or in computersoftware, firmware, or hardware, including the structures disclosed inthis specification and their structural equivalents, or in combinationsof one or more of them. The subject matter described in thisspecification can be implemented as one or more computer programs, e.g.,one or more circuits of computer program instructions, encoded on one ormore computer storage media for execution by, or to control theoperation of, data processing apparatuses. Alternatively or in addition,the program instructions can be encoded on an artificially generatedpropagated signal, e.g., a machine-generated electrical, optical, orelectromagnetic signal that is generated to encode information fortransmission to suitable receiver apparatus for execution by a dataprocessing apparatus. A computer storage medium can be, or be includedin, a computer-readable storage device, a computer-readable storagesubstrate, a random or serial access memory array or device, or acombination of one or more of them. While a computer storage medium isnot a propagated signal, a computer storage medium can be a source ordestination of computer program instructions encoded in an artificiallygenerated propagated signal. The computer storage medium can also be, orbe included in, one or more separate components or media (e.g., multipleCDs, disks, or other storage devices). The operations described in thisspecification can be implemented as operations performed by a dataprocessing apparatus on data stored on one or more computer-readablestorage devices or received from other sources.

The terms “data processing system” “computing device” “component” or“data processing apparatus” encompass various apparatuses, devices, andmachines for processing data, including by way of example a programmableprocessor, a computer, a system on a chip, or multiple ones, orcombinations of the foregoing. The apparatus can include special purposelogic circuitry, e.g., an FPGA (field programmable gate array) or anASIC (application specific integrated circuit). The apparatus can alsoinclude, in addition to hardware, code that creates an executionenvironment for the computer program in question, e.g., code thatconstitutes processor firmware, a protocol stack, a database managementsystem, an operating system, a cross-platform runtime environment, avirtual machine, or a combination of one or more of them. The apparatusand execution environment can realize various different computing modelinfrastructures, such as web services, distributed computing and gridcomputing infrastructures. The digital assistant component 106, andother data processing system 102 or remote data processing system 136components can include or share one or more data processing apparatuses,systems, computing devices, or processors. The digital assistant server138 and content selector component 140 can include or share one or moredata processing apparatuses, systems, computing devices, or processors.

A computer program (also known as a program, software, softwareapplication, app, script, or code) can be written in any form ofprogramming language, including compiled or interpreted languages,declarative or procedural languages, and can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, object, or other unit suitable for use in a computingenvironment. A computer program can correspond to a file in a filesystem. A computer program can be stored in a portion of a file thatholds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs (e.g., components of the data processing system 102)to perform actions by operating on input data and generating output. Theprocesses and logic flows can also be performed by, and apparatuses canalso be implemented as, special purpose logic circuitry, e.g., an FPGA(field programmable gate array) or an ASIC (application specificintegrated circuit). Devices suitable for storing computer programinstructions and data include all forms of non-volatile memory, mediaand memory devices, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto optical disks; andCD ROM and DVD-ROM disks. The processor and the memory can besupplemented by, or incorporated in, special purpose logic circuitry.

The subject matter described herein can be implemented in a computingsystem that includes a back end component, e.g., as a data server, orthat includes a middleware component, e.g., an application server, orthat includes a front end component, e.g., a client computer having agraphical user interface or a web browser through which a user caninteract with an implementation of the subject matter described in thisspecification, or a combination of one or more such back end,middleware, or front end components. The components of the system can beinterconnected by any form or medium of digital data communication,e.g., a communication network. Examples of communication networksinclude a local area network (“LAN”) and a wide area network (“WAN”), aninter-network (e.g., the Internet), and peer-to-peer networks (e.g., adhoc peer-to-peer networks).

The computing system such as system 100 or system 700 can includeclients and servers. A client and server are generally remote from eachother and typically interact through a communication network (e.g., thenetwork 105). The relationship of client and server arises by virtue ofcomputer programs running on the respective computers and having aclient-server relationship to each other. In some implementations, aserver transmits data (e.g., data packets representing a digitalcomponent) to a client device (e.g., for purposes of displaying data toand receiving user input from a user interacting with the clientdevice). Data generated at the client device (e.g., a result of the userinteraction) can be received from the client device at the server (e.g.,received by the digital assistant server 138 from the digital assistantcomponent 106 of the computing device 122 or the 3P digital contentprovider device 134).

While operations are depicted in the drawings in a particular order,such operations are not required to be performed in the particular ordershown or in sequential order, and all illustrated operations are notrequired to be performed. Actions described herein can be performed in adifferent order.

The separation of various system components does not require separationin all implementations, and the described program components can beincluded in a single hardware or software product. For example, thedynamic notifier component 116, and the digital assistant selectorcomponent 120 can be a single component, app, or program, or a logicdevice having one or more processing circuits, or executed by one ormore processors of the data processing system 102.

Having now described some illustrative implementations, it is apparentthat the foregoing is illustrative and not limiting, having beenpresented by way of example. In particular, although many of theexamples presented herein involve specific combinations of method actsor system elements, those acts and those elements may be combined inother ways to accomplish the same objectives. Acts, elements andfeatures discussed in connection with one implementation are notintended to be excluded from a similar role in other implementations orimplementations.

The phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including” “comprising” “having” “containing” “involving”“characterized by” “characterized in that” and variations thereofherein, is meant to encompass the items listed thereafter, equivalentsthereof, and additional items, as well as alternate implementationsconsisting of the items listed thereafter exclusively. In oneimplementation, the systems and methods described herein consist of one,each combination of more than one, or all of the described elements,acts, or components.

Any references to implementations or elements or acts of the systems andmethods herein referred to in the singular may also embraceimplementations including a plurality of these elements, and anyreferences in plural to any implementation or element or act herein mayalso embrace implementations including only a single element. Referencesin the singular or plural form are not intended to limit the presentlydisclosed systems or methods, their components, acts, or elements tosingle or plural configurations. References to any act or element beingbased on any information, act or element may include implementationswhere the act or element is based at least in part on any information,act, or element.

Any implementation disclosed herein may be combined with any otherimplementation or embodiment, and references to “an implementation,”“some implementations,” “one implementation” or the like are notnecessarily mutually exclusive and are intended to indicate that aparticular feature, structure, or characteristic described in connectionwith the implementation may be included in at least one implementationor embodiment. Such terms as used herein are not necessarily allreferring to the same implementation. Any implementation may be combinedwith any other implementation, inclusively or exclusively, in any mannerconsistent with the aspects and implementations disclosed herein.

References to “or” may be construed as inclusive so that any termsdescribed using “or” may indicate any of a single, more than one, andall of the described terms. References to at least one of a conjunctivelist of terms may be construed as an inclusive OR to indicate any of asingle, more than one, and all of the described terms. For example, areference to “at least one of ‘A’ and ‘B’” can include only ‘A’, only‘B’, as well as both ‘A’ and ‘B’. Such references used in conjunctionwith “comprising” or other open terminology can include additionalitems.

Where technical features in the drawings, detailed description or anyclaim are followed by reference signs, the reference signs have beenincluded to increase the intelligibility of the drawings, detaileddescription, and claims. Accordingly, neither the reference signs northeir absence have any limiting effect on the scope of any claimelements.

The systems and methods described herein may be embodied in otherspecific forms without departing from the characteristics thereof. Forexample, devices, products or services described as 3P or third partysuch as the 3P digital content provider device 134 can be or include,partially or entirely, first party devices, products or services, andcan be commonly owned by an entity associated with the data processingsystem 102, the digital assistant server 138, or other components. Theforegoing implementations are illustrative rather than limiting of thedescribed systems and methods. Scope of the systems and methodsdescribed herein is thus indicated by the appended claims, rather thanthe foregoing description, and changes that come within the meaning andrange of equivalency of the claims are embraced therein.

What is claimed is:
 1. A system to adjust execution of digital actions,comprising: a data processing system comprising one or more processorsto: load, via a client computing device of a plurality of clientcomputing devices linked to an electronic account, a script libraryembedded in an electronic resource, the script library comprising aplurality of call-to-actions for the electronic resource configured forexecution by one or more digital assistants provided by each of theplurality of client computing devices; query a digital assistantcomponent to determine an historic level of engagement between theplurality of client computing devices and the one or more digitalassistants; select, based on a first property of the client computingdevice and the historic level of engagement, a type of digital interfacein which to present a call-to-action of the plurality ofcall-to-actions; generate, based on the type of digital interfaceselected based on the first property of the client computing device andthe historic level of engagement, a digital interface with thecall-to-action; detect, via the digital interface, an instruction toexecute the call-to-action; determine, responsive to the instruction toexecute the call-to-action, a mode of execution for the call-to-actionbased on a second property of the client computing device and thehistoric level of engagement; select, based on the mode of execution, adigital assistant from the one or more digital assistants and a secondclient device of the plurality of client computing devices to executethe call-to-action; and transmit the call-to-action to the second clientdevice to cause the second client device to invoke the digital assistantto execute the call-to-action.
 2. The system of claim 1, wherein thescript library is established by a third-party service provider with theplurality of call-to-actions comprising a predetermined set ofcall-to-actions selected by the third-party service provider.
 3. Thesystem of claim 1, comprising: the data processing system to load agraphical user interface element corresponding to the script library ata location on the electronic resource established by a provider of theelectronic resource.
 4. The system of claim 1, comprising: the dataprocessing system to receive, via a network from the digital assistantcomponent remote from the data processing system, information about thehistoric level of engagement.
 5. The system of claim 1, comprising: thedata processing system to determine the historic level of engagementbased on a number of previous interactions between the plurality ofclient computing devices during a time interval.
 6. The system of claim1, comprising: the data processing system to determine the historiclevel of engagement for a type of call-to-action.
 7. The system of claim1, comprising: the data processing system to select the type of digitalinterface comprising a voice-based interface based on the first propertyof the client computing device indicating the client computing devicecomprises a microphone and a speaker, and the historic level ofengagement indicating a number of interactions between the plurality ofclient computing devices and the one or more digital assistants greaterthan a threshold.
 8. The system of claim 1, comprising: the dataprocessing system to select the type of digital interface comprising agraphical user interface based on the first property of the clientcomputing device indicating the client computing device lacks amicrophone or access to the microphone or a speaker of the clientcomputing device by the one or more digital assistants has beendisabled.
 9. The system of claim 1, comprising: the data processingsystem to select the mode of execution for the call-to-action based onthe second property of the client computing device indicating a type ofoperating system of the client computing device.
 10. The system of claim1, wherein the first property indicates the client computing device is asmartphone enabled with the digital assistant, and the historic level ofengagement indicates that the plurality of client computing devicesinteracted with the one or more digital assistants during a timeinterval prior to the script library loaded by the data processingsystem for the electronic resource, comprising: the data processingsystem to select the digital interface comprising a pop-up icon overlaidon the electronic resource.
 11. A method of adjusting execution ofdigital actions, comprising: loading, by a data processing systemcomprising one or more processors, via a client computing device of aplurality of client computing devices linked to an electronic account, ascript library embedded in an electronic resource, the script librarycomprising a plurality of call-to-actions for the electronic resourceconfigured for execution by one or more digital assistants provided byeach of the plurality of client computing devices; querying, by the dataprocessing system, a digital assistant component to determine anhistoric level of engagement between the plurality of client computingdevices and the one or more digital assistants; selecting, by the dataprocessing system based on a first property of the client computingdevice and the historic level of engagement, a type of digital interfacein which to present a call-to-action of the plurality ofcall-to-actions; generating, by the data processing system based on thetype of digital interface selected based on the first property of theclient computing device and the historic level of engagement, a digitalinterface with the call-to-action; detecting, by the data processingsystem via the digital interface, an instruction to execute thecall-to-action; determining, by the data processing system responsive tothe instruction to execute the call-to-action, a mode of execution forthe call-to-action based on a second property of the client computingdevice and the historic level of engagement; selecting, by the dataprocessing system based on the mode of execution, a digital assistantfrom the one or more digital assistants and a second client device ofthe plurality of client computing devices to execute the call-to-action;and transmitting, by the data processing system, the call-to-action tothe second client device to cause the second client device to invoke thedigital assistant to execute the call-to-action.
 12. The method of claim11, wherein the script library is established by a third-party serviceprovider with the plurality of call-to-actions comprising apredetermined set of call-to-actions selected by the third-party serviceprovider.
 13. The method of claim 11, comprising: loading a graphicaluser interface element corresponding to the script library at a locationon the electronic resource established by a provider of the electronicresource.
 14. The method of claim 11, comprising: receiving via anetwork from the digital assistant component remote from the dataprocessing system, information about the historic level of engagement.15. The method of claim 11, comprising: determining the historic levelof engagement based on a number of previous interactions between theplurality of client computing devices during a time interval.
 16. Themethod of claim 11, comprising: determining the historic level ofengagement for a type of call-to-action.
 17. The method of claim 11,comprising: selecting the type of digital interface comprising avoice-based interface based on the first property of the clientcomputing device indicating the client computing device comprises amicrophone and a speaker, and the historic level of engagementindicating a number of interactions between the plurality of clientcomputing devices and the one or more digital assistants greater than athreshold.
 18. The method of claim 11, comprising: selecting the type ofdigital interface comprising a graphical user interface based on thefirst property of the client computing device indicating the clientcomputing device lacks a microphone or access to the microphone or aspeaker of the client computing device by the one or more digitalassistants has been disabled.
 19. The method of claim 11, comprising:selecting the mode of execution for the call-to-action based on thesecond property of the client computing device indicating a type ofoperating system of the client computing device.
 20. The method of claim11, wherein the first property indicates the client computing device isa smartphone enabled with the digital assistant, and the historic levelof engagement indicates that the plurality of client computing devicesinteracted with the one or more digital assistants during a timeinterval prior to the script library loaded by the data processingsystem for the electronic resource, comprising: selecting the digitalinterface comprising a pop-up icon overlaid on the electronic resource.